-- Logs begin at Mon 2022-10-03 11:56:17 UTC, end at Mon 2022-10-03 13:09:57 UTC. -- Oct 03 12:29:18.557100 dv-cinder-844563-12-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 03 12:29:18.566085 dv-cinder-844563-12-iscsi devstack@keystone.service[81794]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 03 12:29:18.568473 dv-cinder-844563-12-iscsi devstack@keystone.service[81794]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Oct 3 12:29:18 2022] *** Oct 03 12:29:18.568473 dv-cinder-844563-12-iscsi devstack@keystone.service[81794]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 03 12:29:18.568473 dv-cinder-844563-12-iscsi devstack@keystone.service[81794]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 03 12:29:18.568473 dv-cinder-844563-12-iscsi devstack@keystone.service[81794]: nodename: dv-cinder-844563-12-iscsi Oct 03 12:29:18.568473 dv-cinder-844563-12-iscsi devstack@keystone.service[81794]: machine: x86_64 Oct 03 12:29:18.568473 dv-cinder-844563-12-iscsi devstack@keystone.service[81794]: clock source: unix Oct 03 12:29:18.568473 dv-cinder-844563-12-iscsi devstack@keystone.service[81794]: pcre jit disabled Oct 03 12:29:18.568473 dv-cinder-844563-12-iscsi devstack@keystone.service[81794]: detected number of CPU cores: 4 Oct 03 12:29:18.568473 dv-cinder-844563-12-iscsi devstack@keystone.service[81794]: current working directory: / Oct 03 12:29:18.568473 dv-cinder-844563-12-iscsi devstack@keystone.service[81794]: detected binary path: /usr/bin/uwsgi-core Oct 03 12:29:18.568473 dv-cinder-844563-12-iscsi devstack@keystone.service[81794]: your processes number limit is 47680 Oct 03 12:29:18.568473 dv-cinder-844563-12-iscsi devstack@keystone.service[81794]: your memory page size is 4096 bytes Oct 03 12:29:18.568473 dv-cinder-844563-12-iscsi devstack@keystone.service[81794]: detected max file descriptor number: 2048 Oct 03 12:29:18.568473 dv-cinder-844563-12-iscsi devstack@keystone.service[81794]: lock engine: pthread robust mutexes Oct 03 12:29:18.568473 dv-cinder-844563-12-iscsi devstack@keystone.service[81794]: thunder lock: enabled Oct 03 12:29:18.569519 dv-cinder-844563-12-iscsi devstack@keystone.service[81794]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 03 12:29:18.569519 dv-cinder-844563-12-iscsi devstack@keystone.service[81794]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 03 12:29:18.608901 dv-cinder-844563-12-iscsi devstack@keystone.service[81794]: Python main interpreter initialized at 0x561d21d479c0 Oct 03 12:29:18.608901 dv-cinder-844563-12-iscsi devstack@keystone.service[81794]: python threads support enabled Oct 03 12:29:18.608901 dv-cinder-844563-12-iscsi devstack@keystone.service[81794]: your server socket listen backlog is limited to 100 connections Oct 03 12:29:18.608901 dv-cinder-844563-12-iscsi devstack@keystone.service[81794]: your mercy for graceful operations on workers is 90 seconds Oct 03 12:29:18.609277 dv-cinder-844563-12-iscsi devstack@keystone.service[81794]: mapped 403077 bytes (393 KB) for 2 cores Oct 03 12:29:18.609381 dv-cinder-844563-12-iscsi devstack@keystone.service[81794]: *** Operational MODE: preforking *** Oct 03 12:29:18.609654 dv-cinder-844563-12-iscsi devstack@keystone.service[81794]: *** uWSGI is running in multiple interpreter mode *** Oct 03 12:29:18.609654 dv-cinder-844563-12-iscsi devstack@keystone.service[81794]: spawned uWSGI master process (pid: 81794) Oct 03 12:29:18.609856 dv-cinder-844563-12-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 03 12:29:18.610928 dv-cinder-844563-12-iscsi devstack@keystone.service[81794]: spawned uWSGI worker 1 (pid: 81805, cores: 1) Oct 03 12:29:18.610928 dv-cinder-844563-12-iscsi devstack@keystone.service[81794]: spawned uWSGI worker 2 (pid: 81806, cores: 1) Oct 03 12:29:18.610928 dv-cinder-844563-12-iscsi devstack@keystone.service[81794]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 03 12:29:23.045157 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 03 12:29:23.045157 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: from cryptography.utils import int_from_bytes Oct 03 12:29:23.045940 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 03 12:29:23.045940 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: from cryptography.utils import int_from_bytes Oct 03 12:29:23.046441 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 03 12:29:23.046441 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: from cryptography.utils import int_from_bytes Oct 03 12:29:23.046890 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 03 12:29:23.046890 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: from cryptography.utils import int_from_bytes Oct 03 12:29:23.097000 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 03 12:29:23.097195 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 03 12:29:23.097404 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] command line args: [] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 03 12:29:23.097504 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 03 12:29:23.097640 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 03 12:29:23.097723 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 03 12:29:23.097839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] command line args: [] {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 03 12:29:23.097971 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ================================================================================ {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 03 12:29:23.098114 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] admin_token = **** {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.098201 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 03 12:29:23.098287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ================================================================================ {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 03 12:29:23.098365 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] config_dir = [] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.098534 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] admin_token = **** {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.098618 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.098712 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] config_source = [] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.098816 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] config_dir = [] {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.099787 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] debug = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.099883 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.099883 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] config_source = [] {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.100006 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.100243 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] debug = True {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.100243 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.100396 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.100462 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.100526 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.100585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.100734 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.100817 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.100948 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.101028 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.101182 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] list_limit = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.101273 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] list_limit = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.101346 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] log_config_append = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.101411 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] log_config_append = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.101524 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.101606 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.101700 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] log_dir = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.101771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] log_dir = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.101879 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] log_file = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.101958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] log_file = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.102115 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.102202 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.102367 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.102446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.102617 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.102706 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.102871 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.102997 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.103115 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.103242 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.103389 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.103509 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.103624 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.103743 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.103864 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.103990 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.104112 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.104231 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.104351 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.104471 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.104599 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.104710 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.104823 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.104941 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.105059 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.105188 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.105315 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.105438 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.105557 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.105672 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.105792 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.105914 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.106025 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] publish_errors = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.106139 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] publish_errors = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.106262 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.106443 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.106570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.106681 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.106799 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.106913 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.107092 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.107229 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.107361 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.107489 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.107610 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] standard_threads = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.107735 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] standard_threads = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.107856 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.107970 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.108088 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.108208 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.108329 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.108443 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.108556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] use_journal = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.108675 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] use_journal = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.108795 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] use_json = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.108914 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] use_json = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.109028 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] use_stderr = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.109149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] use_stderr = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.109263 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] use_syslog = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.109376 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] use_syslog = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.109489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.109608 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 12:29:23.109728 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.109849 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.109964 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.110077 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.110196 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.110309 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.110498 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.110613 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.110733 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.110853 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.110966 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.111078 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.111199 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.111328 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.111445 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.111558 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.111671 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.111783 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.111908 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.112023 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.112136 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.112248 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.112361 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.112475 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.112600 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.112714 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.112833 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.112960 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.113080 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.113198 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.113321 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.113436 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.113557 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.113671 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.113784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.113896 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.114021 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.114140 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.114254 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.114367 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.114944 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.115079 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.115212 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.115369 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.115489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.115616 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.115738 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.115852 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.115974 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.116095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.116216 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.116330 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.116443 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.116556 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.116785 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.116905 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.117025 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.117145 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.117266 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.117381 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.117529 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.117650 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.117771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.117869 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.117986 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.118106 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.118231 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.118351 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.118465 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.118577 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.118691 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.118813 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.118928 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.119040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.119040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.119215 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.119310 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.119442 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.119541 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.119642 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.119737 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.119831 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.119925 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.120027 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.120129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.120225 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.120339 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.120458 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.120610 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.120731 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.120858 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.120963 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.121082 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.121203 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.121313 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.121438 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.121553 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.121667 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.121780 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.121901 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.122022 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.122145 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.122269 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.122390 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.122504 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.122618 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.122731 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.122834 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.122959 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.123075 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.123197 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.123354 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.123473 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.123699 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.123830 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.123951 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.124168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.124295 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.124411 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.124525 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.124648 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.124770 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.124886 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.125001 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.125119 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.125242 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.125366 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.125489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.125611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.125726 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.125840 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.125954 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.126068 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.126191 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.126306 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.126421 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.126534 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.126653 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.126768 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.126890 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.127005 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.127120 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.127240 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.127369 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.127485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.127613 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.127730 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.127844 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.127964 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.128078 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.128200 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.128414 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.128541 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.128657 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.128772 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.128886 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.128999 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.129123 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.129286 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.129393 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.129489 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.129602 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.129716 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.129843 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.129971 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.130086 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.130560 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.130678 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.130792 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.130906 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.131031 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.131147 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.131262 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.131428 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.131545 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.131660 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.131660 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.131898 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.132014 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.132128 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.132242 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.132356 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.132483 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.132599 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.132713 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.132827 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.132941 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.133055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.133178 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.133294 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.133409 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.133528 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.133650 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.133765 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.133888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.134011 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.134126 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.134240 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.134354 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.134468 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.134591 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.134706 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.134820 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.134933 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.135046 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.135210 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.135349 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.135480 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.135599 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.135719 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.135835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.135949 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.136073 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.136220 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.136350 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] auth.external = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.136467 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] auth.external = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.136670 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.136778 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.136883 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.136982 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.137088 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.137187 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.137283 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] auth.password = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.137378 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] auth.password = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.137475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] auth.token = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.137570 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] auth.token = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.137675 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.137864 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.137962 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.138068 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.138164 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.138311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.138416 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.138519 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.138615 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.138711 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.138807 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.138911 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.139019 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.139121 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.139219 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] credential.caching = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.139343 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] credential.caching = True {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.141935 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.142052 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.142233 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.142314 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.142388 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.142461 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.142533 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.142615 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.142697 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.142772 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.142846 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.142976 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.143056 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.143139 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.143214 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.143287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.143383 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.143459 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.143532 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.143618 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.143695 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.143768 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.143840 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.143913 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.143991 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.144139 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.144216 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.144289 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.144361 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.144433 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.144506 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] federation.caching = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.144587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] federation.caching = True {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.144663 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.144736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.144808 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.144880 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.144953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.145034 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.145109 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.145182 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.145254 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.145327 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.145399 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.145480 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.145555 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.145628 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.146620 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.146720 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.146798 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.146881 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.146957 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.147037 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.147112 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.147192 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.147266 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] identity.caching = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.147433 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] identity.caching = True {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.147513 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.147586 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.147664 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.147743 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.147816 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.147897 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.147980 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.148054 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.148177 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.148255 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.148328 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.148409 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.148484 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.148556 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.148628 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.148707 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.148789 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.148863 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.148935 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.149007 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.149079 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.149151 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.149232 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.149306 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.149378 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.149449 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.149521 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.149593 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.149673 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.149748 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.149820 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.149892 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.149964 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.150035 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.150116 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.150190 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.150263 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.150334 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.150406 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.150477 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.150560 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.150635 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.150708 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.150781 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.150852 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.150930 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.151048 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.151126 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.151199 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.151260 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.151347 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.151424 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.151424 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.151576 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.151651 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.151723 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.151795 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.151866 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.151946 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.152021 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.152093 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.152164 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.152235 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.152307 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.152387 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.152469 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.152543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.152619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.152691 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.152763 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.152843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.152917 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.152989 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.153068 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.153142 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.153214 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.153294 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.153408 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.153589 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.153664 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.153664 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.153795 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.153878 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.153953 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.154025 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.154097 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.154170 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.154242 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.154323 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.154397 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.154469 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.154541 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.154668 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.154749 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.154907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.154988 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.155146 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.155226 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.155409 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.155493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.155644 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.user = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.155725 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.user = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.155885 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.155964 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.156141 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.156220 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.156371 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.156452 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.156618 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.156696 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.156863 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.156942 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.157107 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.157186 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.157342 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.157420 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.157572 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.157660 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.157817 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.157902 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.158050 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.158124 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.158293 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.158410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.158558 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.158702 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.158826 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.158961 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.159103 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.159239 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.159385 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.159510 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.159640 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.159767 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.159895 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.160033 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.160159 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.160285 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.160412 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.160540 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.160669 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.160804 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.160942 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.161070 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.161204 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.162038 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.162497 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.162638 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.162771 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.162901 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.163030 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.163158 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.163287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.163423 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.163502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.163575 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.163648 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.163721 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.163794 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.163942 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.164019 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.164099 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.164214 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.164293 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.164366 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.164519 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.164599 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.164724 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.164843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.164959 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.165120 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.165251 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.165398 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.165497 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.165594 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] resource.caching = True {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.165689 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] resource.caching = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.165785 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.165890 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.165990 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.166087 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.166183 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.166280 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.166377 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.166481 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.166580 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.166676 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.166771 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.166867 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.166964 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.167072 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.167171 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.167267 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.167385 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.167488 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.167585 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] role.caching = True {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.170621 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] role.caching = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.170729 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] role.driver = sql {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.170905 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] role.driver = sql {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.170984 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.171059 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.171132 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.171215 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.171291 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.171435 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.171511 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.171584 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.171656 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.171743 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.171819 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.171892 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.171965 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.172037 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.172110 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.172192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.172267 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.172340 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.172417 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.172491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.172572 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.172647 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.172719 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.172792 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.172864 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.172936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.173016 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.173091 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.173164 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.173236 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.173309 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.173381 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.173461 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.173536 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.173609 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.173682 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.173754 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.173826 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.173907 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.173982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.174055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.174127 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.174200 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.174273 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.174353 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.174428 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.174500 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.174572 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.174678 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.174802 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.174889 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.174965 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.175038 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.175110 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.175182 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.175254 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.175423 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.175502 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.175576 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.175648 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.175720 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.175793 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.175882 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.175958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.176031 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.176103 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.176175 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] token.caching = True {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.176256 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] token.caching = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.176330 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.176403 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.176475 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.176596 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.176673 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.176755 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.176829 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.176829 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.177388 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.177480 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.177556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.177640 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.177716 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.177788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.177861 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.177933 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.178006 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.178087 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.178162 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.178235 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.178360 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.178438 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.178511 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.178592 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.178720 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.178801 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.178971 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.179050 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.179268 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.179383 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.179560 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.179653 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.179807 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.179889 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] database.connection = **** {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.180056 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.180135 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.180306 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.180387 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.connection = **** {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.180550 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.180631 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.180793 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.180988 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.181106 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.181221 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.181351 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.181469 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.181584 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.181701 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.181818 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.181934 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.182123 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.182243 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.182359 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.182474 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.182590 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.182707 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.182831 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.182949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.183115 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.183238 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.183398 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.183517 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.183647 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.183766 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.183882 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.183997 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.184113 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.184229 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.184353 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.184469 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.184584 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81806) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 03 12:29:23.184700 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.184700 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 12:29:23.184700 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81805) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 03 12:29:23.215077 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.215298 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.215548 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.218321 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.218557 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.218780 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.224142 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.227203 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.252081 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.252325 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.253352 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.253584 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.331225 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.331413 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.379786 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.380562 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.381519 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.381754 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.459352 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.459577 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.459794 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.460019 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.460243 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.460464 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.535013 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.535245 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.535491 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.535713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.535933 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.536152 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.556292 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.556516 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.556741 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.556960 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.557175 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.557387 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.557602 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.570223 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.582904 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.583056 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.583242 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.583384 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.583508 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.583648 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.583725 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.583841 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.583936 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.584058 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.584142 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.584272 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.584357 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.584543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81806) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.592631 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.595450 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.595685 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.595906 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.596123 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.596336 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.596548 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.596758 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81805) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 12:29:23.601579 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.601789 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 03 12:29:23.602134 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.602336 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 03 12:29:23.602612 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.602875 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.603138 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.603427 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.607085 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.607298 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 03 12:29:23.607662 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.607864 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 03 12:29:23.608146 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.608412 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.608728 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.608992 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.612993 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.613275 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.613548 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.613835 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.619424 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.619706 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.619980 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.620268 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.621343 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81806) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 12:29:23.624255 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81806) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 12:29:23.626887 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.626887 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.626887 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.626887 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.626887 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.626887 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.626887 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.627419 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.627419 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.627419 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.627624 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81805) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 12:29:23.631069 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81805) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 12:29:23.631384 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.631671 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.631998 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.632288 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.632551 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.632821 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.633096 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.633368 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.633755 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.634030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.650273 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.651762 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81806) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 12:29:23.652052 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.656724 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81806) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 12:29:23.657008 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.657089 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.657290 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.658520 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81805) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 12:29:23.658807 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.663663 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81806) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 12:29:23.663762 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81805) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 12:29:23.663955 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.664038 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.664297 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.669591 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81806) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 12:29:23.669892 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.670184 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.670472 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.670677 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81805) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 12:29:23.670772 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.670970 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.671051 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.671269 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.671567 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.675166 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81805) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 12:29:23.675485 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.675777 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.676064 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.676339 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.676615 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.677411 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.677783 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.687712 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.687996 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.691851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81806) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 12:29:23.694010 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.694287 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.695741 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.696040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.698237 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81805) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 12:29:23.700268 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81806) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 12:29:23.701536 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.701842 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.703433 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81806) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 12:29:23.706100 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81805) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 12:29:23.706636 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.706941 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.707230 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.707605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.707906 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.708292 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.709694 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81805) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 12:29:23.713008 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.713311 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.713599 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.714136 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.714420 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.714710 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.724402 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81806) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 12:29:23.724693 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.724963 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.725227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.731977 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81805) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 12:29:23.732077 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.732267 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.732539 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.732804 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.733617 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.735146 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.735474 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.738159 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81806) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 12:29:23.738450 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.738737 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.739026 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.739959 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.740259 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.741808 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.743598 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.743887 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.746566 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81805) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 12:29:23.746853 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.747245 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.747566 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.752949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.753036 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81806) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 12:29:23.753036 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.753036 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.753036 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.753036 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.753036 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.753479 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.759939 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81805) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 12:29:23.760230 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.760613 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.760899 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.761185 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.761467 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.761757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.768739 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81806) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 12:29:23.771689 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81806) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 12:29:23.774565 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.776023 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.777253 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81805) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 12:29:23.777669 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81806) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 12:29:23.777959 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.778353 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.780220 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81805) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 12:29:23.783185 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.784639 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.784766 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.786235 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81805) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 12:29:23.786362 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81806) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 12:29:23.786859 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.786859 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.789278 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.789680 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.789971 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.790256 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.798204 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.798204 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81805) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 12:29:23.798204 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.798204 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.798670 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81806) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 12:29:23.798751 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.798825 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.798899 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.798973 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.806179 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81806) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 12:29:23.806468 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.806752 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.807041 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.807168 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81805) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 12:29:23.807337 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.807503 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.807619 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.807789 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.807895 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.808174 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.808445 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.808722 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.809019 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.809291 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.809559 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.809835 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81806) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.815073 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81805) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 12:29:23.815447 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.815664 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.815945 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.816231 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.816517 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.816796 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.817069 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.817343 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.818221 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.818677 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.818769 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.819015 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.819285 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81805) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 12:29:23.844800 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x561d21d479c0 pid: 81806 (default app) Oct 03 12:29:23.848050 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9935d8-0e10-4263-a6a9-b644a0ae4f5e None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:23.848341 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9935d8-0e10-4263-a6a9-b644a0ae4f5e None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:23.848615 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9935d8-0e10-4263-a6a9-b644a0ae4f5e None None] PATH_INFO: `/v3` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:23.852809 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1/1] 10.222.0.33 () {50 vars in 791 bytes} [Mon Oct 3 12:29:23 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:29:23.854395 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x561d21d479c0 pid: 81805 (default app) Oct 03 12:29:31.502659 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c7720-3e26-4452-93a3-567baade786f None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:31.503144 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c7720-3e26-4452-93a3-567baade786f None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:31.503144 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c7720-3e26-4452-93a3-567baade786f None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:31.504510 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 2/2] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:29:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 12:29:31.512513 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fe372-87c9-464e-a9ea-e46d8d049dc6 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:31.512959 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fe372-87c9-464e-a9ea-e46d8d049dc6 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:31.513057 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fe372-87c9-464e-a9ea-e46d8d049dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:31.584919 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: INFO dbcounter [None req-7e1fe372-87c9-464e-a9ea-e46d8d049dc6 None None] Registered counter for database keystone Oct 03 12:29:31.604582 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG oslo_db.sqlalchemy.engines [None req-7e1fe372-87c9-464e-a9ea-e46d8d049dc6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81805) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 03 12:29:31.610842 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writer thread running {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 03 12:29:31.787346 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG passlib.handlers.bcrypt [None req-7e1fe372-87c9-464e-a9ea-e46d8d049dc6 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81805) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 03 12:29:31.787790 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG passlib.handlers.bcrypt [None req-7e1fe372-87c9-464e-a9ea-e46d8d049dc6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81805) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 03 12:29:31.845689 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-7e1fe372-87c9-464e-a9ea-e46d8d049dc6 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:29:31.847277 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-7e1fe372-87c9-464e-a9ea-e46d8d049dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:29:31.976470 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1/3] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:29:31 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 467 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 03 12:29:32.459611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c7720-3e26-4452-93a3-567baade786f None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:32.532933 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: INFO dbcounter [None req-c35c7720-3e26-4452-93a3-567baade786f None None] Registered counter for database keystone Oct 03 12:29:32.555295 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG oslo_db.sqlalchemy.engines [None req-c35c7720-3e26-4452-93a3-567baade786f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81806) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 03 12:29:32.561583 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writer thread running {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 03 12:29:32.682901 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c7720-3e26-4452-93a3-567baade786f None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:32.687344 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d176184a-617f-4fe5-9862-e0af1453e402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:32.687995 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176184a-617f-4fe5-9862-e0af1453e402 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:32.688281 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176184a-617f-4fe5-9862-e0af1453e402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:32.688559 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176184a-617f-4fe5-9862-e0af1453e402 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:32.697451 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d176184a-617f-4fe5-9862-e0af1453e402 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:32.750202 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d176184a-617f-4fe5-9862-e0af1453e402 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 03 12:29:32.750555 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d176184a-617f-4fe5-9862-e0af1453e402 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:32.760367 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 3/4] 10.222.0.33 () {58 vars in 1324 bytes} [Mon Oct 3 12:29:32 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 12:29:32.766324 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1fe372-87c9-464e-a9ea-e46d8d049dc6 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:32.783509 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1fe372-87c9-464e-a9ea-e46d8d049dc6 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:32.788453 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7800637e-af9c-4774-966e-b0c240d9cddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:32.789699 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800637e-af9c-4774-966e-b0c240d9cddd admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:32.789986 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800637e-af9c-4774-966e-b0c240d9cddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:32.790264 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800637e-af9c-4774-966e-b0c240d9cddd admin admin] PATH_INFO: `/v3/projects/912de75d08d24473bf98cc746c209242` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:32.791770 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7800637e-af9c-4774-966e-b0c240d9cddd admin admin] RBAC: Authorizing `identity:get_project(project_id=912de75d08d24473bf98cc746c209242)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:32.843114 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7800637e-af9c-4774-966e-b0c240d9cddd admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 03 12:29:32.843494 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7800637e-af9c-4774-966e-b0c240d9cddd admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:32.847956 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 2/5] 10.222.0.33 () {56 vars in 1207 bytes} [Mon Oct 3 12:29:32 2022] GET /identity/v3/projects/912de75d08d24473bf98cc746c209242 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:29:34.984189 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf8991-e711-4e0f-9579-c32c15a220aa None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:34.984731 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf8991-e711-4e0f-9579-c32c15a220aa None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:34.984731 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf8991-e711-4e0f-9579-c32c15a220aa None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:34.988810 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 4/6] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:29:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 12:29:34.994465 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1f6c6-bcea-47b5-b11f-8b6c53b1c608 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:34.994873 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1f6c6-bcea-47b5-b11f-8b6c53b1c608 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:34.995011 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1f6c6-bcea-47b5-b11f-8b6c53b1c608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:35.014767 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-7fd1f6c6-bcea-47b5-b11f-8b6c53b1c608 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:29:35.016396 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-7fd1f6c6-bcea-47b5-b11f-8b6c53b1c608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:29:35.024768 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 3/7] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:29:34 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 03 12:29:35.291389 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf8991-e711-4e0f-9579-c32c15a220aa None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:35.304119 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf8991-e711-4e0f-9579-c32c15a220aa None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:35.308298 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc3b26-943e-48f4-b89d-a198c7ab5640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:35.308893 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc3b26-943e-48f4-b89d-a198c7ab5640 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:35.309173 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc3b26-943e-48f4-b89d-a198c7ab5640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:35.309444 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc3b26-943e-48f4-b89d-a198c7ab5640 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:35.317332 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bc3b26-943e-48f4-b89d-a198c7ab5640 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:35.319814 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bc3b26-943e-48f4-b89d-a198c7ab5640 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:35.328678 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 5/8] 10.222.0.33 () {58 vars in 1324 bytes} [Mon Oct 3 12:29:35 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 03 12:29:35.334850 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd1f6c6-bcea-47b5-b11f-8b6c53b1c608 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:35.346633 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd1f6c6-bcea-47b5-b11f-8b6c53b1c608 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:35.351489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74d0377-d7ad-4a4f-80f6-af25d9acfe75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:35.352114 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d0377-d7ad-4a4f-80f6-af25d9acfe75 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:35.352393 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d0377-d7ad-4a4f-80f6-af25d9acfe75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:35.352664 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d0377-d7ad-4a4f-80f6-af25d9acfe75 admin admin] PATH_INFO: `/v3/users/77a14102256040ab81672f6adeaa6c5a` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:35.353961 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74d0377-d7ad-4a4f-80f6-af25d9acfe75 admin admin] RBAC: Authorizing `identity:get_user(user_id=77a14102256040ab81672f6adeaa6c5a)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:35.355830 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74d0377-d7ad-4a4f-80f6-af25d9acfe75 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:35.357332 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 4/9] 10.222.0.33 () {56 vars in 1198 bytes} [Mon Oct 3 12:29:35 2022] GET /identity/v3/users/77a14102256040ab81672f6adeaa6c5a => generated 247 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:29:37.548817 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56493dcf-f374-4c75-b868-803261e4da4d None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:37.549320 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56493dcf-f374-4c75-b868-803261e4da4d None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:37.549320 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56493dcf-f374-4c75-b868-803261e4da4d None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:37.550593 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 6/10] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:29:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:29:37.556000 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ee812-242f-4a77-9650-8bbe24148a38 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:37.556381 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ee812-242f-4a77-9650-8bbe24148a38 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:37.556666 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ee812-242f-4a77-9650-8bbe24148a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:37.574924 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-815ee812-242f-4a77-9650-8bbe24148a38 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:29:37.576131 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-815ee812-242f-4a77-9650-8bbe24148a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:29:37.607546 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3510c-c217-4b54-91d6-2ffbf9767539 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:37.607829 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3510c-c217-4b54-91d6-2ffbf9767539 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:37.608106 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3510c-c217-4b54-91d6-2ffbf9767539 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:37.609423 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 7/11] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:29:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 12:29:37.614684 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada050c-c84b-4040-9739-2c747ad278e2 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:37.614968 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada050c-c84b-4040-9739-2c747ad278e2 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:37.615278 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada050c-c84b-4040-9739-2c747ad278e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:37.657223 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG passlib.handlers.bcrypt [None req-eada050c-c84b-4040-9739-2c747ad278e2 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81806) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 03 12:29:37.657604 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG passlib.handlers.bcrypt [None req-eada050c-c84b-4040-9739-2c747ad278e2 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81806) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 03 12:29:37.660821 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 5/12] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:29:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:29:37.717385 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-eada050c-c84b-4040-9739-2c747ad278e2 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:29:37.719011 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-eada050c-c84b-4040-9739-2c747ad278e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:29:37.817991 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 8/13] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:29:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 12:29:37.930346 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ee812-242f-4a77-9650-8bbe24148a38 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:37.954859 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:38.048429 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ee812-242f-4a77-9650-8bbe24148a38 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:38.082647 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9178d221-b9dc-4bd3-9f41-c3e1c62cd613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:38.083209 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178d221-b9dc-4bd3-9f41-c3e1c62cd613 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:38.083515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178d221-b9dc-4bd3-9f41-c3e1c62cd613 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:38.083788 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178d221-b9dc-4bd3-9f41-c3e1c62cd613 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:38.095185 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9178d221-b9dc-4bd3-9f41-c3e1c62cd613 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:29:38.096221 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada050c-c84b-4040-9739-2c747ad278e2 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:38.105676 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-9178d221-b9dc-4bd3-9f41-c3e1c62cd613 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:29:38.105676 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:29:38.105676 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:29:38.105676 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:29:38.105676 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:29:38.105676 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:29:38.105676 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:29:38.105676 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:29:38.105676 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:29:38.105676 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:29:38.105676 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:29:38.105676 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:29:38.105676 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:29:38.105676 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:29:38.105676 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:29:38.105676 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:29:38.106705 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:29:38.106705 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:29:38.106705 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:29:38.106705 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:29:38.106705 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:29:38.106705 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:29:38.106705 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:29:38.106705 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:29:38.106705 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:29:38.106705 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:29:38.106705 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:29:38.106705 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:29:38.106705 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:29:38.106705 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:29:38.106705 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:29:38.106705 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:29:38.108695 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:29:38.108695 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:29:38.108695 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:29:38.108695 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:29:38.108695 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:29:38.108695 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:29:38.108695 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:29:38.108695 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:29:38.108695 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:29:38.108695 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:29:38.108695 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:29:38.108695 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:29:38.108695 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 6/14] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 12:29:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:29:38.112424 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9178d221-b9dc-4bd3-9f41-c3e1c62cd613 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:38.194601 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:38.197814 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada050c-c84b-4040-9739-2c747ad278e2 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:38.200079 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:38.212272 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9178d221-b9dc-4bd3-9f41-c3e1c62cd613 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:38.236138 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e9b685-d859-4183-b4d0-18a18749070b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:38.236138 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e9b685-d859-4183-b4d0-18a18749070b None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:38.236138 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e9b685-d859-4183-b4d0-18a18749070b None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:38.236138 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e9b685-d859-4183-b4d0-18a18749070b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:38.250637 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cebfb3-96df-4a3e-999d-610bc83e391c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:38.251253 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cebfb3-96df-4a3e-999d-610bc83e391c None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:38.251568 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cebfb3-96df-4a3e-999d-610bc83e391c None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:38.251839 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cebfb3-96df-4a3e-999d-610bc83e391c None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:38.252790 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cebfb3-96df-4a3e-999d-610bc83e391c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:29:38.253094 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cebfb3-96df-4a3e-999d-610bc83e391c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:38.254485 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cebfb3-96df-4a3e-999d-610bc83e391c None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:38.266105 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 7/15] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:29:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:29:38.271774 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cebfb3-96df-4a3e-999d-610bc83e391c None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:38.303438 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e9b685-d859-4183-b4d0-18a18749070b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:38.304951 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e9b685-d859-4183-b4d0-18a18749070b None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:38.371009 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cebfb3-96df-4a3e-999d-610bc83e391c None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:38.400653 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 9/16] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:29:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 03 12:29:38.403675 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c604a80-5e6c-4f02-8ab1-748635434815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:38.404317 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c604a80-5e6c-4f02-8ab1-748635434815 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:38.404598 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c604a80-5e6c-4f02-8ab1-748635434815 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:38.404871 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c604a80-5e6c-4f02-8ab1-748635434815 None admin] PATH_INFO: `/v3/users/77a14102256040ab81672f6adeaa6c5a` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:38.406011 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c604a80-5e6c-4f02-8ab1-748635434815 None admin] RBAC: Authorizing `identity:get_user(user_id=77a14102256040ab81672f6adeaa6c5a)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:38.406124 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e9b685-d859-4183-b4d0-18a18749070b None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:38.407507 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c604a80-5e6c-4f02-8ab1-748635434815 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:38.408842 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:38.409044 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 8/17] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:29:38 2022] GET /identity/v3/users/77a14102256040ab81672f6adeaa6c5a => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:29:38.414928 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c604a80-5e6c-4f02-8ab1-748635434815 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:38.430072 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:38.508308 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e9b685-d859-4183-b4d0-18a18749070b None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:38.514527 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c604a80-5e6c-4f02-8ab1-748635434815 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:38.542792 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa098475-48ca-4aba-bfc8-fb09af559dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:38.543687 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa098475-48ca-4aba-bfc8-fb09af559dfd None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:38.543962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa098475-48ca-4aba-bfc8-fb09af559dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:38.544236 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa098475-48ca-4aba-bfc8-fb09af559dfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:38.548986 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db54f1a-152c-4934-b813-8e58b5977272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:38.549619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db54f1a-152c-4934-b813-8e58b5977272 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:38.549897 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db54f1a-152c-4934-b813-8e58b5977272 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:38.550167 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db54f1a-152c-4934-b813-8e58b5977272 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:38.551276 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db54f1a-152c-4934-b813-8e58b5977272 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:38.553234 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db54f1a-152c-4934-b813-8e58b5977272 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:38.554767 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 9/18] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:29:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:29:38.560440 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db54f1a-152c-4934-b813-8e58b5977272 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:38.563273 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa098475-48ca-4aba-bfc8-fb09af559dfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:38.564780 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa098475-48ca-4aba-bfc8-fb09af559dfd None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:38.583149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-aa098475-48ca-4aba-bfc8-fb09af559dfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:29:38.583149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:29:38.583149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:29:38.583149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:29:38.583149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:29:38.583149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:29:38.583149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:29:38.583149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:29:38.583149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:29:38.583149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:29:38.583149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:29:38.583149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:29:38.583149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:29:38.583149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:29:38.583149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:29:38.583149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:29:38.584146 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:29:38.584146 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:29:38.584146 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:29:38.584146 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:29:38.584146 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:29:38.584146 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:29:38.584146 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:29:38.584146 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:29:38.584146 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:29:38.584146 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:29:38.584146 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:29:38.584146 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:29:38.584146 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:29:38.584146 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:29:38.584146 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:29:38.584146 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:29:38.584146 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:29:38.585185 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:29:38.585185 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:29:38.585185 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:29:38.585185 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 10/19] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:29:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:29:38.590323 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa098475-48ca-4aba-bfc8-fb09af559dfd None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:38.653968 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:38.654218 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:38.661789 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db54f1a-152c-4934-b813-8e58b5977272 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:38.688884 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa098475-48ca-4aba-bfc8-fb09af559dfd None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:38.700239 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00301ada-406c-44b9-8dd1-8efd8bd20af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:38.707848 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00301ada-406c-44b9-8dd1-8efd8bd20af3 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:38.707848 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00301ada-406c-44b9-8dd1-8efd8bd20af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:38.707848 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00301ada-406c-44b9-8dd1-8efd8bd20af3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:38.707848 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00301ada-406c-44b9-8dd1-8efd8bd20af3 None admin] RBAC: Adding query filter params (role.id=7933c45dcdd54ef0a98018ba8a8bddaa, scope.domain.id=default, user.id=77a14102256040ab81672f6adeaa6c5a) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:29:38.707848 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00301ada-406c-44b9-8dd1-8efd8bd20af3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:38.707848 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00301ada-406c-44b9-8dd1-8efd8bd20af3 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:38.710902 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 10/20] 10.222.0.33 () {56 vars in 1322 bytes} [Mon Oct 3 12:29:38 2022] GET /identity/v3/role_assignments?user.id=77a14102256040ab81672f6adeaa6c5a&scope.domain.id=default&role.id=7933c45dcdd54ef0a98018ba8a8bddaa => generated 233 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:29:38.726363 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e25e0d-9510-44ea-af6a-a7c69354c731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:38.727156 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e25e0d-9510-44ea-af6a-a7c69354c731 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:38.728078 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e25e0d-9510-44ea-af6a-a7c69354c731 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:38.728361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e25e0d-9510-44ea-af6a-a7c69354c731 None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:38.729046 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e25e0d-9510-44ea-af6a-a7c69354c731 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:29:38.729391 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e25e0d-9510-44ea-af6a-a7c69354c731 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:38.730860 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e25e0d-9510-44ea-af6a-a7c69354c731 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:38.751154 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 11/21] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:29:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:29:41.105827 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878265a-e8a1-4354-a141-b33879e3e850 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:41.106377 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878265a-e8a1-4354-a141-b33879e3e850 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:41.106377 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878265a-e8a1-4354-a141-b33879e3e850 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:41.107768 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 11/22] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:29:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:29:41.113108 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458bc288-cf8f-4a03-8e50-482538c317d1 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:41.113543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458bc288-cf8f-4a03-8e50-482538c317d1 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:41.113720 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458bc288-cf8f-4a03-8e50-482538c317d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:41.133681 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-458bc288-cf8f-4a03-8e50-482538c317d1 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:29:41.134887 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-458bc288-cf8f-4a03-8e50-482538c317d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:29:41.211432 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 12/23] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:29:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 12:29:41.428617 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8301f-980c-4f04-976b-b5e89a9731b2 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:41.428797 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8301f-980c-4f04-976b-b5e89a9731b2 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:41.428991 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8301f-980c-4f04-976b-b5e89a9731b2 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:41.430851 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 12/24] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:29:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:29:41.436469 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83ed80-3a0a-4c00-a923-d12498ec13a1 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:41.436987 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83ed80-3a0a-4c00-a923-d12498ec13a1 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:41.437491 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83ed80-3a0a-4c00-a923-d12498ec13a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:41.464545 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-1a83ed80-3a0a-4c00-a923-d12498ec13a1 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:29:41.466063 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-1a83ed80-3a0a-4c00-a923-d12498ec13a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:29:41.521212 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:41.550344 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 13/25] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:29:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:29:41.596030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8301f-980c-4f04-976b-b5e89a9731b2 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:41.715041 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8301f-980c-4f04-976b-b5e89a9731b2 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:41.751565 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b0ef8-8040-46b0-aa6c-4b8824016b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:41.752568 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b0ef8-8040-46b0-aa6c-4b8824016b0a None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:41.753099 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b0ef8-8040-46b0-aa6c-4b8824016b0a None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:41.753620 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b0ef8-8040-46b0-aa6c-4b8824016b0a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:41.764491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b0ef8-8040-46b0-aa6c-4b8824016b0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:29:41.769634 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-496b0ef8-8040-46b0-aa6c-4b8824016b0a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:29:41.769634 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:29:41.769634 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:29:41.769634 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:29:41.769634 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:29:41.769634 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:29:41.769634 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:29:41.769634 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:29:41.769634 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:29:41.769634 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:29:41.769634 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:29:41.769634 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:29:41.769634 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:29:41.769634 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:29:41.769634 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:29:41.769634 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:29:41.770492 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:29:41.770492 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:29:41.770492 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:29:41.770492 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:29:41.770492 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:29:41.770492 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:29:41.770492 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:29:41.770492 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:29:41.770492 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:29:41.770492 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:29:41.770492 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:29:41.770492 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:29:41.770492 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:29:41.770492 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:29:41.770492 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:29:41.770492 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:29:41.771247 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:29:41.771247 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:29:41.771247 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:29:41.771247 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:29:41.771247 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:29:41.771247 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:29:41.771247 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:29:41.771247 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:29:41.771247 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:29:41.771247 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:29:41.771247 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:29:41.771247 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:29:41.774564 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 13/26] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 12:29:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:29:41.780390 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83ed80-3a0a-4c00-a923-d12498ec13a1 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:41.931068 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83ed80-3a0a-4c00-a923-d12498ec13a1 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:41.978446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8198596f-6ab2-4989-a84d-05251807f92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:41.978995 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198596f-6ab2-4989-a84d-05251807f92f None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:41.979283 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198596f-6ab2-4989-a84d-05251807f92f None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:41.979586 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198596f-6ab2-4989-a84d-05251807f92f None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:41.980271 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8198596f-6ab2-4989-a84d-05251807f92f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:29:41.980582 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8198596f-6ab2-4989-a84d-05251807f92f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:41.982013 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8198596f-6ab2-4989-a84d-05251807f92f None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:42.003815 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 14/27] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:29:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:29:42.011091 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b0ef8-8040-46b0-aa6c-4b8824016b0a None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:42.015164 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8198596f-6ab2-4989-a84d-05251807f92f None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:42.017520 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:42.153524 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b0ef8-8040-46b0-aa6c-4b8824016b0a None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:42.154634 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:42.192242 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c9e87f-e296-4617-9902-164fc7815803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:42.194116 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9e87f-e296-4617-9902-164fc7815803 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:42.194643 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9e87f-e296-4617-9902-164fc7815803 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:42.195139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9e87f-e296-4617-9902-164fc7815803 None admin] PATH_INFO: `/v3/users/77a14102256040ab81672f6adeaa6c5a` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:42.196544 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c9e87f-e296-4617-9902-164fc7815803 None admin] RBAC: Authorizing `identity:get_user(user_id=77a14102256040ab81672f6adeaa6c5a)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:42.200215 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c9e87f-e296-4617-9902-164fc7815803 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:42.200942 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 14/28] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:29:42 2022] GET /identity/v3/users/77a14102256040ab81672f6adeaa6c5a => generated 247 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:29:42.206592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9391b3-e3f8-499b-89b9-a9ba03d71885 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:42.207092 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9391b3-e3f8-499b-89b9-a9ba03d71885 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:42.207572 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9391b3-e3f8-499b-89b9-a9ba03d71885 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:42.211098 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 15/29] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:29:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:29:42.212221 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9391b3-e3f8-499b-89b9-a9ba03d71885 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:42.261567 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8198596f-6ab2-4989-a84d-05251807f92f None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:42.299083 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709094ef-7fa7-4b28-8dc4-2610f2917877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:42.300119 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709094ef-7fa7-4b28-8dc4-2610f2917877 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:42.300581 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709094ef-7fa7-4b28-8dc4-2610f2917877 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:42.301033 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709094ef-7fa7-4b28-8dc4-2610f2917877 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:42.313181 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709094ef-7fa7-4b28-8dc4-2610f2917877 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:42.314896 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709094ef-7fa7-4b28-8dc4-2610f2917877 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:42.335299 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-709094ef-7fa7-4b28-8dc4-2610f2917877 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:29:42.335299 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:29:42.335299 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:29:42.335299 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:29:42.335299 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:29:42.335299 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:29:42.335299 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:29:42.335299 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:29:42.335299 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:29:42.335299 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:29:42.335299 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:29:42.335299 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:29:42.335299 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:29:42.335299 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:29:42.335299 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:29:42.335299 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:29:42.336208 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:29:42.336208 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:29:42.336208 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:29:42.336208 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:29:42.336208 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:29:42.336208 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:29:42.336208 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:29:42.336208 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:29:42.336208 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:29:42.336208 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:29:42.336208 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:29:42.336208 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:29:42.336208 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:29:42.336208 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:29:42.336208 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:29:42.336208 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:29:42.336208 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:29:42.337017 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:29:42.337017 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:29:42.337017 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:29:42.338957 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 15/30] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:29:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:29:42.343583 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ac5b9-c195-4a3f-a748-b7dc94d6b22e None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:42.344036 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ac5b9-c195-4a3f-a748-b7dc94d6b22e None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:42.344472 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ac5b9-c195-4a3f-a748-b7dc94d6b22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:42.383717 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-0e2ac5b9-c195-4a3f-a748-b7dc94d6b22e None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:29:42.385166 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-0e2ac5b9-c195-4a3f-a748-b7dc94d6b22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:29:42.410767 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:42.478959 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9391b3-e3f8-499b-89b9-a9ba03d71885 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:42.518703 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a5798-d91b-4386-8583-3636782a1b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:42.519258 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a5798-d91b-4386-8583-3636782a1b8b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:42.519566 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a5798-d91b-4386-8583-3636782a1b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:42.519845 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a5798-d91b-4386-8583-3636782a1b8b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:42.521313 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1a5798-d91b-4386-8583-3636782a1b8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:42.523630 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1a5798-d91b-4386-8583-3636782a1b8b None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:42.526794 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 16/31] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:29:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:29:42.527791 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a5798-d91b-4386-8583-3636782a1b8b None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:42.596409 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 16/32] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:29:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:29:42.597268 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ac5b9-c195-4a3f-a748-b7dc94d6b22e None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:42.641941 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:42.690157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a5798-d91b-4386-8583-3636782a1b8b None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:42.736908 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ac5b9-c195-4a3f-a748-b7dc94d6b22e None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:42.774926 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e11b07-7594-4d47-a358-2b7c7ddc2263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:42.775883 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e11b07-7594-4d47-a358-2b7c7ddc2263 None admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:42.776173 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e11b07-7594-4d47-a358-2b7c7ddc2263 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:42.776457 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e11b07-7594-4d47-a358-2b7c7ddc2263 None admin] PATH_INFO: `/v3/domains/default/users/77a14102256040ab81672f6adeaa6c5a/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:42.779051 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e11b07-7594-4d47-a358-2b7c7ddc2263 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=77a14102256040ab81672f6adeaa6c5a, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:42.780556 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e11b07-7594-4d47-a358-2b7c7ddc2263 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:42.787185 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c732db-b2b3-4142-9245-d319fe7a5797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:42.787876 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c732db-b2b3-4142-9245-d319fe7a5797 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:42.788163 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c732db-b2b3-4142-9245-d319fe7a5797 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:42.788447 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c732db-b2b3-4142-9245-d319fe7a5797 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:42.789051 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c732db-b2b3-4142-9245-d319fe7a5797 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:29:42.789357 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c732db-b2b3-4142-9245-d319fe7a5797 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:42.790736 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c732db-b2b3-4142-9245-d319fe7a5797 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:42.805883 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:42.809847 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 17/33] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:29:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:29:42.811077 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f4ad4-eda0-48bf-a6e6-5c7948749ce7 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:42.816077 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f4ad4-eda0-48bf-a6e6-5c7948749ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:42.816439 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f4ad4-eda0-48bf-a6e6-5c7948749ce7 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:42.818169 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 18/34] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:29:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:29:42.819556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f4ad4-eda0-48bf-a6e6-5c7948749ce7 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:42.947442 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f4ad4-eda0-48bf-a6e6-5c7948749ce7 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:42.983213 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c6afb1-2c55-4fa1-8b58-d6e6e3109f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:42.983909 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c6afb1-2c55-4fa1-8b58-d6e6e3109f25 None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:42.984187 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c6afb1-2c55-4fa1-8b58-d6e6e3109f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:42.984460 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c6afb1-2c55-4fa1-8b58-d6e6e3109f25 None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:42.985110 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c6afb1-2c55-4fa1-8b58-d6e6e3109f25 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:42.986541 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c6afb1-2c55-4fa1-8b58-d6e6e3109f25 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:43.042329 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 17/35] 10.222.0.33 () {58 vars in 1361 bytes} [Mon Oct 3 12:29:42 2022] PUT /identity/v3/domains/default/users/77a14102256040ab81672f6adeaa6c5a/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:29:43.044148 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da799125-8756-43c4-816d-a1ce2c9f9120 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:43.044446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da799125-8756-43c4-816d-a1ce2c9f9120 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:43.044726 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da799125-8756-43c4-816d-a1ce2c9f9120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:43.068962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-da799125-8756-43c4-816d-a1ce2c9f9120 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:29:43.070608 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-da799125-8756-43c4-816d-a1ce2c9f9120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:29:43.153789 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 18/36] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:29:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:29:43.154762 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da799125-8756-43c4-816d-a1ce2c9f9120 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:43.175842 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:43.269300 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 19/37] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Oct 3 12:29:42 2022] POST /identity/v3/projects => generated 301 bytes in 450 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:29:43.270897 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b91b69-b2dc-49a0-82be-83a2ac9c486e None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:43.271187 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b91b69-b2dc-49a0-82be-83a2ac9c486e None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:43.271667 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b91b69-b2dc-49a0-82be-83a2ac9c486e None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:43.272977 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 20/38] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:29:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:29:43.274001 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba798da3-1007-4d3d-bba7-edb9e7f49c3f None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:43.274368 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba798da3-1007-4d3d-bba7-edb9e7f49c3f None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:43.274703 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba798da3-1007-4d3d-bba7-edb9e7f49c3f None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:43.276304 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 21/39] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:29:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:29:43.278670 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc9c70-5e71-40d1-bb1b-7616f33207d0 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:43.278955 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc9c70-5e71-40d1-bb1b-7616f33207d0 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:43.279229 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc9c70-5e71-40d1-bb1b-7616f33207d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:43.284122 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da799125-8756-43c4-816d-a1ce2c9f9120 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:43.299284 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-7bcc9c70-5e71-40d1-bb1b-7616f33207d0 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:29:43.300986 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-7bcc9c70-5e71-40d1-bb1b-7616f33207d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:29:43.337273 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26dd1f3-3c44-472a-b042-f0af6b49356a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:43.338143 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26dd1f3-3c44-472a-b042-f0af6b49356a None admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:43.339636 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26dd1f3-3c44-472a-b042-f0af6b49356a None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:43.340037 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26dd1f3-3c44-472a-b042-f0af6b49356a None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:43.341005 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26dd1f3-3c44-472a-b042-f0af6b49356a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:43.344564 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26dd1f3-3c44-472a-b042-f0af6b49356a None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:43.390034 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:43.416407 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 22/40] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:29:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:29:43.418041 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c19c37-10ca-46e6-9375-9903368bbcb7 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:43.418336 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c19c37-10ca-46e6-9375-9903368bbcb7 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:43.418617 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c19c37-10ca-46e6-9375-9903368bbcb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:43.439656 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-01c19c37-10ca-46e6-9375-9903368bbcb7 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:29:43.440959 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-01c19c37-10ca-46e6-9375-9903368bbcb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:29:43.487793 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 19/41] 10.222.0.33 () {60 vars in 1131 bytes} [Mon Oct 3 12:29:43 2022] POST /identity/v3/roles => generated 219 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:29:43.488658 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26dd1f3-3c44-472a-b042-f0af6b49356a None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:43.696101 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 23/42] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:29:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:29:43.699736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:43.721489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c19c37-10ca-46e6-9375-9903368bbcb7 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:43.752200 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26dd1f3-3c44-472a-b042-f0af6b49356a None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:43.812333 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5973cd76-37fe-416b-be74-849e05df2b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:43.812964 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5973cd76-37fe-416b-be74-849e05df2b3c None admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:43.813246 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5973cd76-37fe-416b-be74-849e05df2b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:43.813519 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5973cd76-37fe-416b-be74-849e05df2b3c None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:43.814196 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5973cd76-37fe-416b-be74-849e05df2b3c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:43.816129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5973cd76-37fe-416b-be74-849e05df2b3c None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:43.876927 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c19c37-10ca-46e6-9375-9903368bbcb7 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:43.893239 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:43.902280 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 20/43] 10.222.0.33 () {60 vars in 1131 bytes} [Mon Oct 3 12:29:43 2022] POST /identity/v3/roles => generated 225 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:29:43.926667 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaf2dbd-228f-4f29-ad7e-d8683ff08b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:43.928453 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf2dbd-228f-4f29-ad7e-d8683ff08b8a None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:43.928959 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf2dbd-228f-4f29-ad7e-d8683ff08b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:43.929495 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf2dbd-228f-4f29-ad7e-d8683ff08b8a None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:43.930530 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaf2dbd-228f-4f29-ad7e-d8683ff08b8a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:43.932471 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaf2dbd-228f-4f29-ad7e-d8683ff08b8a None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:44.006884 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5973cd76-37fe-416b-be74-849e05df2b3c None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:44.038100 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 24/44] 10.222.0.33 () {60 vars in 1131 bytes} [Mon Oct 3 12:29:43 2022] POST /identity/v3/roles => generated 223 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:29:44.134172 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5973cd76-37fe-416b-be74-849e05df2b3c None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:44.168135 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:44.175757 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb44ef-eefb-41bb-8e5d-bdee59ae721a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:44.176633 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb44ef-eefb-41bb-8e5d-bdee59ae721a None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:44.176997 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb44ef-eefb-41bb-8e5d-bdee59ae721a None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:44.177381 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb44ef-eefb-41bb-8e5d-bdee59ae721a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:44.179059 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb44ef-eefb-41bb-8e5d-bdee59ae721a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:44.180625 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb44ef-eefb-41bb-8e5d-bdee59ae721a None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:44.182727 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 21/45] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:29:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:29:44.188512 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaf2dbd-228f-4f29-ad7e-d8683ff08b8a None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:44.305952 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaf2dbd-228f-4f29-ad7e-d8683ff08b8a None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:44.340485 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b65b9b1-4653-4e55-bf87-d7f897dde2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:44.341103 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b65b9b1-4653-4e55-bf87-d7f897dde2d7 None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:44.341386 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b65b9b1-4653-4e55-bf87-d7f897dde2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:44.341663 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b65b9b1-4653-4e55-bf87-d7f897dde2d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:44.342316 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b65b9b1-4653-4e55-bf87-d7f897dde2d7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:44.343816 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b65b9b1-4653-4e55-bf87-d7f897dde2d7 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:44.434376 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 25/46] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Oct 3 12:29:44 2022] POST /identity/v3/projects => generated 312 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:29:45.474237 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6065c-f92c-4e52-a9a6-72077dbb2bdd None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:45.474806 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6065c-f92c-4e52-a9a6-72077dbb2bdd None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:45.475024 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6065c-f92c-4e52-a9a6-72077dbb2bdd None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:45.476420 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 22/47] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:29:45.481691 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a551cda-6edd-469e-af79-10ee7ad270a9 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:45.482085 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a551cda-6edd-469e-af79-10ee7ad270a9 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:45.482252 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a551cda-6edd-469e-af79-10ee7ad270a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:45.500844 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-7a551cda-6edd-469e-af79-10ee7ad270a9 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:29:45.501960 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-7a551cda-6edd-469e-af79-10ee7ad270a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:29:45.519451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:45.567662 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 26/48] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:29:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:29:45.835183 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f6065c-f92c-4e52-a9a6-72077dbb2bdd None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:45.934874 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f6065c-f92c-4e52-a9a6-72077dbb2bdd None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:45.968792 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f9d16-973a-40f3-9175-5c0b7c6bd93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:45.969422 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f9d16-973a-40f3-9175-5c0b7c6bd93a None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:45.969705 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f9d16-973a-40f3-9175-5c0b7c6bd93a None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:45.969979 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f9d16-973a-40f3-9175-5c0b7c6bd93a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:45.981003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f9d16-973a-40f3-9175-5c0b7c6bd93a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:29:45.987978 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-114f9d16-973a-40f3-9175-5c0b7c6bd93a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:29:45.987978 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:29:45.987978 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:29:45.987978 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:29:45.987978 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:29:45.987978 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:29:45.987978 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:29:45.987978 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:29:45.987978 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:29:45.987978 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:29:45.987978 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:29:45.987978 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:29:45.987978 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:29:45.987978 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:29:45.987978 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:29:45.987978 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:29:45.989084 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:29:45.989084 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:29:45.989084 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:29:45.989084 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:29:45.989084 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:29:45.989084 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:29:45.989084 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:29:45.989084 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:29:45.989084 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:29:45.989084 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:29:45.989084 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:29:45.989084 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:29:45.989084 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:29:45.989084 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:29:45.989084 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:29:45.989084 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:29:45.991227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:29:45.991227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:29:45.991227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:29:45.991227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:29:45.991227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:29:45.991227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:29:45.991227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:29:45.991227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:29:45.991227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:29:45.991227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:29:45.991227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:29:45.991227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:29:45.991227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 23/49] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 12:29:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:29:45.994876 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a551cda-6edd-469e-af79-10ee7ad270a9 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:46.093874 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a551cda-6edd-469e-af79-10ee7ad270a9 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:46.129735 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8304afda-addd-4a42-9c29-bbf252b99575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:46.130464 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304afda-addd-4a42-9c29-bbf252b99575 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:46.131659 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304afda-addd-4a42-9c29-bbf252b99575 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:46.132440 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304afda-addd-4a42-9c29-bbf252b99575 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:46.133125 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8304afda-addd-4a42-9c29-bbf252b99575 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:29:46.133436 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8304afda-addd-4a42-9c29-bbf252b99575 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:46.134873 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8304afda-addd-4a42-9c29-bbf252b99575 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:46.143924 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 27/50] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:29:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:29:46.150018 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f9d16-973a-40f3-9175-5c0b7c6bd93a None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:46.317643 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:46.360782 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f9d16-973a-40f3-9175-5c0b7c6bd93a None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:46.393817 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9347b582-0e9d-46a7-8018-053c0739e659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:46.394473 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347b582-0e9d-46a7-8018-053c0739e659 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:46.394759 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347b582-0e9d-46a7-8018-053c0739e659 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:46.395037 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347b582-0e9d-46a7-8018-053c0739e659 None admin] PATH_INFO: `/v3/users/77a14102256040ab81672f6adeaa6c5a` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:46.396215 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9347b582-0e9d-46a7-8018-053c0739e659 None admin] RBAC: Authorizing `identity:get_user(user_id=77a14102256040ab81672f6adeaa6c5a)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:46.398830 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9347b582-0e9d-46a7-8018-053c0739e659 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:46.400454 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 24/51] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:29:46 2022] GET /identity/v3/users/77a14102256040ab81672f6adeaa6c5a => generated 247 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:29:46.405883 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8304afda-addd-4a42-9c29-bbf252b99575 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:46.425573 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:46.511373 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8304afda-addd-4a42-9c29-bbf252b99575 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:46.544974 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ebcd17-5b53-495b-820e-0445f5699fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:46.545609 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ebcd17-5b53-495b-820e-0445f5699fe2 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:46.545892 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ebcd17-5b53-495b-820e-0445f5699fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:46.546172 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ebcd17-5b53-495b-820e-0445f5699fe2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:46.547273 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ebcd17-5b53-495b-820e-0445f5699fe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:46.548736 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ebcd17-5b53-495b-820e-0445f5699fe2 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:46.550304 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 28/52] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:29:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:29:46.556120 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9347b582-0e9d-46a7-8018-053c0739e659 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:46.585420 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3de2a5-20b9-4693-a665-d79bd6ae4f04 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:46.585707 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3de2a5-20b9-4693-a665-d79bd6ae4f04 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:46.585991 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3de2a5-20b9-4693-a665-d79bd6ae4f04 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:46.587346 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 29/53] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 12:29:46.592629 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace79252-0bb2-4827-bf4e-02dfded9ec02 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:46.592911 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace79252-0bb2-4827-bf4e-02dfded9ec02 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:46.593189 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace79252-0bb2-4827-bf4e-02dfded9ec02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:46.610647 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-ace79252-0bb2-4827-bf4e-02dfded9ec02 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:29:46.611925 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ace79252-0bb2-4827-bf4e-02dfded9ec02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:29:46.656114 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9347b582-0e9d-46a7-8018-053c0739e659 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:46.677154 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 30/54] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:29:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 12:29:46.680723 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:46.688973 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af534e-bfd6-450e-b3cc-6c3f62170dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:46.703440 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af534e-bfd6-450e-b3cc-6c3f62170dda None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:46.703440 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af534e-bfd6-450e-b3cc-6c3f62170dda None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:46.703440 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af534e-bfd6-450e-b3cc-6c3f62170dda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:46.703440 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8af534e-bfd6-450e-b3cc-6c3f62170dda None admin] RBAC: Adding query filter params (role.id=7933c45dcdd54ef0a98018ba8a8bddaa, scope.domain.id=default, user.id=77a14102256040ab81672f6adeaa6c5a) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:29:46.703440 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8af534e-bfd6-450e-b3cc-6c3f62170dda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:46.703440 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8af534e-bfd6-450e-b3cc-6c3f62170dda None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:46.704157 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 25/55] 10.222.0.33 () {56 vars in 1322 bytes} [Mon Oct 3 12:29:46 2022] GET /identity/v3/role_assignments?user.id=77a14102256040ab81672f6adeaa6c5a&scope.domain.id=default&role.id=7933c45dcdd54ef0a98018ba8a8bddaa => generated 530 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:29:46.949047 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace79252-0bb2-4827-bf4e-02dfded9ec02 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:46.963796 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:47.044468 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace79252-0bb2-4827-bf4e-02dfded9ec02 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:47.077387 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73003e8a-2136-4998-9b48-629f58fd520b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:47.078015 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73003e8a-2136-4998-9b48-629f58fd520b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:47.078298 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73003e8a-2136-4998-9b48-629f58fd520b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:47.078573 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73003e8a-2136-4998-9b48-629f58fd520b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:47.079690 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73003e8a-2136-4998-9b48-629f58fd520b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:47.081141 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73003e8a-2136-4998-9b48-629f58fd520b None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:47.082674 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 31/56] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:29:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:29:47.088213 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af534e-bfd6-450e-b3cc-6c3f62170dda None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:47.186985 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af534e-bfd6-450e-b3cc-6c3f62170dda None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:47.219379 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eebc82-8480-495c-93d8-4fb2e5cb26b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:47.219952 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eebc82-8480-495c-93d8-4fb2e5cb26b9 None admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:47.220263 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eebc82-8480-495c-93d8-4fb2e5cb26b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:47.220524 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eebc82-8480-495c-93d8-4fb2e5cb26b9 None admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:47.221179 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eebc82-8480-495c-93d8-4fb2e5cb26b9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:47.222627 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eebc82-8480-495c-93d8-4fb2e5cb26b9 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:47.308484 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 26/57] 10.222.0.33 () {60 vars in 1140 bytes} [Mon Oct 3 12:29:47 2022] POST /identity/v3/projects => generated 298 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:29:49.556160 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2b92e-c0e6-4ca2-913b-445723d5d370 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:49.556644 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2b92e-c0e6-4ca2-913b-445723d5d370 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:49.556644 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2b92e-c0e6-4ca2-913b-445723d5d370 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:49.558134 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 32/58] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:29:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:29:49.563188 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bba485-746d-4e66-8b18-7363e976345d None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:49.563629 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bba485-746d-4e66-8b18-7363e976345d None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:49.563786 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bba485-746d-4e66-8b18-7363e976345d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:49.582131 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-46bba485-746d-4e66-8b18-7363e976345d None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:29:49.583309 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-46bba485-746d-4e66-8b18-7363e976345d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:29:49.617127 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:49.648175 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 27/59] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:29:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:29:49.934542 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e2b92e-c0e6-4ca2-913b-445723d5d370 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:50.015690 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:50.034467 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e2b92e-c0e6-4ca2-913b-445723d5d370 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:50.068071 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f66af4b-934b-4e68-b783-7d27abf4ea52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:50.068728 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66af4b-934b-4e68-b783-7d27abf4ea52 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:50.069009 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66af4b-934b-4e68-b783-7d27abf4ea52 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:50.069284 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66af4b-934b-4e68-b783-7d27abf4ea52 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:50.070383 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f66af4b-934b-4e68-b783-7d27abf4ea52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:50.071846 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f66af4b-934b-4e68-b783-7d27abf4ea52 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:50.073425 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 33/60] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:29:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:29:50.079044 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bba485-746d-4e66-8b18-7363e976345d None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:50.177117 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bba485-746d-4e66-8b18-7363e976345d None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:50.210210 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2726deb1-f9fb-48d3-967d-45d646a1536b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:50.210760 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726deb1-f9fb-48d3-967d-45d646a1536b None admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:50.211044 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726deb1-f9fb-48d3-967d-45d646a1536b None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:50.211342 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726deb1-f9fb-48d3-967d-45d646a1536b None admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:50.211996 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2726deb1-f9fb-48d3-967d-45d646a1536b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:50.213437 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2726deb1-f9fb-48d3-967d-45d646a1536b None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:50.313428 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 28/61] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Oct 3 12:29:50 2022] POST /identity/v3/users => generated 275 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:29:53.428153 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f50f5-689e-4132-988f-d78e2fb12a5b None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:53.429175 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f50f5-689e-4132-988f-d78e2fb12a5b None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:53.429652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f50f5-689e-4132-988f-d78e2fb12a5b None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:53.431479 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 34/62] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:29:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:29:53.436580 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d938c-c68e-4e2c-9b66-5e4c87e0dbfe None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:53.436968 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d938c-c68e-4e2c-9b66-5e4c87e0dbfe None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:53.437125 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d938c-c68e-4e2c-9b66-5e4c87e0dbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:53.460282 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-8b5d938c-c68e-4e2c-9b66-5e4c87e0dbfe None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:29:53.461471 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-8b5d938c-c68e-4e2c-9b66-5e4c87e0dbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:29:53.489287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:53.562340 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 29/63] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:29:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:29:53.683569 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a63ac-d84f-40c4-b034-870bbeef91b3 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:53.684220 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a63ac-d84f-40c4-b034-870bbeef91b3 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:53.684691 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a63ac-d84f-40c4-b034-870bbeef91b3 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:53.686251 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 35/64] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:29:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:29:53.691639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7780bff-5d88-4c4c-96ed-374e2be134aa None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:53.692168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7780bff-5d88-4c4c-96ed-374e2be134aa None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:53.692651 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7780bff-5d88-4c4c-96ed-374e2be134aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:53.719492 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-f7780bff-5d88-4c4c-96ed-374e2be134aa None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:29:53.720972 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-f7780bff-5d88-4c4c-96ed-374e2be134aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:29:53.816015 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 30/65] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:29:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 12:29:53.856825 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a63ac-d84f-40c4-b034-870bbeef91b3 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:53.999991 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a63ac-d84f-40c4-b034-870bbeef91b3 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:54.009643 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c78508-582c-4e4e-a4e5-ef165509576b None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:54.010230 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c78508-582c-4e4e-a4e5-ef165509576b None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:54.010703 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c78508-582c-4e4e-a4e5-ef165509576b None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:54.012290 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 31/66] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:29:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:29:54.020493 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84673321-fdb0-4c48-b4e7-e3923923a014 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:54.021007 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84673321-fdb0-4c48-b4e7-e3923923a014 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:54.022305 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84673321-fdb0-4c48-b4e7-e3923923a014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:54.048606 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-84673321-fdb0-4c48-b4e7-e3923923a014 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:29:54.051593 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-84673321-fdb0-4c48-b4e7-e3923923a014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:29:54.066860 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e7975-c26b-4b96-8ac1-51ce0f2d31e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:54.067842 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e7975-c26b-4b96-8ac1-51ce0f2d31e0 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:54.068846 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e7975-c26b-4b96-8ac1-51ce0f2d31e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:54.069336 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e7975-c26b-4b96-8ac1-51ce0f2d31e0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:54.088669 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0e7975-c26b-4b96-8ac1-51ce0f2d31e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:29:54.094276 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-6f0e7975-c26b-4b96-8ac1-51ce0f2d31e0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 12:29:54.094276 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:29:54.094276 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:29:54.094276 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:29:54.094276 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:29:54.094276 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:29:54.094276 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:29:54.094276 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:29:54.094276 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:29:54.094276 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:29:54.094276 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:29:54.094276 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:29:54.094276 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:29:54.094276 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:29:54.094276 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:29:54.094276 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:29:54.095094 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:29:54.095094 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:29:54.095094 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:29:54.095094 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:29:54.095094 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:29:54.095094 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:29:54.095094 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:29:54.095094 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:29:54.095094 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:29:54.095094 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:29:54.095094 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:29:54.095094 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:29:54.095094 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:29:54.095094 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:29:54.095094 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:29:54.095094 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:29:54.095880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:29:54.095880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:29:54.095880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:29:54.095880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:29:54.095880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:29:54.095880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:29:54.095880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:29:54.095880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:29:54.095880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:29:54.095880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:29:54.095880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 12:29:54.095880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:29:54.102008 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 36/67] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:29:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:29:54.108339 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e7975-c26b-4b96-8ac1-51ce0f2d31e0 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:54.128180 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:54.193066 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 32/68] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:29:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:29:54.283216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84673321-fdb0-4c48-b4e7-e3923923a014 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:54.298130 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e7975-c26b-4b96-8ac1-51ce0f2d31e0 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:54.330183 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:54.338456 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf4fe7-22fc-4057-b534-de702d628b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:54.339419 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf4fe7-22fc-4057-b534-de702d628b54 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:54.340139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf4fe7-22fc-4057-b534-de702d628b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:54.347659 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf4fe7-22fc-4057-b534-de702d628b54 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:54.347659 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cf4fe7-22fc-4057-b534-de702d628b54 None admin] RBAC: Adding query filter params (name=member) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:29:54.347659 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cf4fe7-22fc-4057-b534-de702d628b54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:54.347659 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cf4fe7-22fc-4057-b534-de702d628b54 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:54.353772 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 37/69] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 12:29:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:29:54.360144 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf4fe7-22fc-4057-b534-de702d628b54 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:54.447143 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84673321-fdb0-4c48-b4e7-e3923923a014 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:54.451287 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:54.489143 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf47b3b-6a54-4d7e-b22d-3a99f7327944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:54.489791 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf47b3b-6a54-4d7e-b22d-3a99f7327944 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:54.490283 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf47b3b-6a54-4d7e-b22d-3a99f7327944 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:54.490398 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf47b3b-6a54-4d7e-b22d-3a99f7327944 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:54.493709 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf4fe7-22fc-4057-b534-de702d628b54 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:54.501487 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf47b3b-6a54-4d7e-b22d-3a99f7327944 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:29:54.506579 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-1cf47b3b-6a54-4d7e-b22d-3a99f7327944 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 12:29:54.506579 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:29:54.506579 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:29:54.506579 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:29:54.506579 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:29:54.506579 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:29:54.506579 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:29:54.506579 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:29:54.506579 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:29:54.506579 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:29:54.506579 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:29:54.506579 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:29:54.506579 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:29:54.506579 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:29:54.506579 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:29:54.506579 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:29:54.507478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:29:54.507478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:29:54.507478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:29:54.507478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:29:54.507478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:29:54.507478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:29:54.507478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:29:54.507478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:29:54.507478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:29:54.507478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:29:54.507478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:29:54.507478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:29:54.507478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:29:54.507478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:29:54.507478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:29:54.507478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:29:54.508305 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:29:54.508305 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:29:54.508305 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:29:54.508305 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:29:54.508305 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:29:54.508305 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:29:54.508305 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:29:54.508305 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:29:54.508305 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:29:54.508305 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:29:54.508305 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 12:29:54.508305 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:29:54.509286 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 33/70] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 12:29:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:29:54.511019 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c4306-e906-4ea5-808c-2aa5ab49dc08 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:54.511310 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c4306-e906-4ea5-808c-2aa5ab49dc08 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:54.511625 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c4306-e906-4ea5-808c-2aa5ab49dc08 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:54.513147 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 34/71] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:29:54.513903 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c4306-e906-4ea5-808c-2aa5ab49dc08 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:54.529691 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbd0d29-f314-45e6-8e79-cae9285bfeb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:54.530332 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd0d29-f314-45e6-8e79-cae9285bfeb6 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:54.530615 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd0d29-f314-45e6-8e79-cae9285bfeb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:54.530895 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd0d29-f314-45e6-8e79-cae9285bfeb6 None admin] PATH_INFO: `/v3/users/b8101b669ca44866a11705e7d0f3b601` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:54.544660 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbd0d29-f314-45e6-8e79-cae9285bfeb6 None admin] RBAC: Authorizing `identity:get_user(user_id=b8101b669ca44866a11705e7d0f3b601)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:54.546107 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbd0d29-f314-45e6-8e79-cae9285bfeb6 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:54.547739 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 38/72] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:29:54 2022] GET /identity/v3/users/b8101b669ca44866a11705e7d0f3b601 => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:29:54.548868 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cca76-a164-442d-9b96-117acc36f697 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:54.549152 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cca76-a164-442d-9b96-117acc36f697 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:54.549429 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cca76-a164-442d-9b96-117acc36f697 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:54.550632 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 39/73] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:29:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:29:54.551344 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03cca76-a164-442d-9b96-117acc36f697 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:54.598177 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:54.620007 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c4306-e906-4ea5-808c-2aa5ab49dc08 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:54.651196 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03cca76-a164-442d-9b96-117acc36f697 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:54.653458 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722c83c7-d863-4031-a083-0cc9cfc951f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:54.654087 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c83c7-d863-4031-a083-0cc9cfc951f6 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:54.654370 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c83c7-d863-4031-a083-0cc9cfc951f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:54.654645 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c83c7-d863-4031-a083-0cc9cfc951f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:54.665030 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722c83c7-d863-4031-a083-0cc9cfc951f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:29:54.669796 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-722c83c7-d863-4031-a083-0cc9cfc951f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:29:54.669796 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:29:54.669796 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:29:54.669796 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:29:54.669796 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:29:54.669796 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:29:54.669796 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:29:54.669796 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:29:54.669796 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:29:54.669796 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:29:54.669796 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:29:54.669796 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:29:54.669796 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:29:54.669796 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:29:54.669796 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:29:54.669796 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:29:54.670795 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:29:54.670795 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:29:54.670795 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:29:54.670795 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:29:54.670795 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:29:54.670795 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:29:54.670795 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:29:54.670795 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:29:54.670795 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:29:54.670795 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:29:54.670795 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:29:54.670795 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:29:54.670795 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:29:54.670795 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:29:54.670795 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:29:54.670795 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:29:54.671603 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:29:54.671603 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:29:54.671603 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:29:54.671603 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:29:54.671603 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:29:54.671603 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:29:54.671603 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:29:54.671603 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:29:54.671603 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:29:54.671603 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:29:54.671603 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:29:54.671603 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:29:54.671603 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 35/74] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 12:29:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:29:54.672663 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999129a-e55a-4716-a6a4-477ce1b5fe20 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:54.672663 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999129a-e55a-4716-a6a4-477ce1b5fe20 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:54.672663 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999129a-e55a-4716-a6a4-477ce1b5fe20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:54.681552 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:54.686921 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaca551-d36a-4bed-879c-dc62447eabaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:54.687581 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaca551-d36a-4bed-879c-dc62447eabaf None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:54.687860 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaca551-d36a-4bed-879c-dc62447eabaf None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:54.692056 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaca551-d36a-4bed-879c-dc62447eabaf None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:54.692056 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaca551-d36a-4bed-879c-dc62447eabaf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:29:54.692056 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaca551-d36a-4bed-879c-dc62447eabaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:54.692056 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaca551-d36a-4bed-879c-dc62447eabaf None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:54.692407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-e999129a-e55a-4716-a6a4-477ce1b5fe20 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:29:54.692407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-e999129a-e55a-4716-a6a4-477ce1b5fe20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:29:54.700355 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 40/75] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:29:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:29:54.701178 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaca551-d36a-4bed-879c-dc62447eabaf None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:54.760518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 36/76] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:29:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:29:54.764593 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a382ff8-86c9-4175-8c6d-52e1a83dae8d None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:54.764883 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a382ff8-86c9-4175-8c6d-52e1a83dae8d None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:54.765218 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a382ff8-86c9-4175-8c6d-52e1a83dae8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:54.783691 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-9a382ff8-86c9-4175-8c6d-52e1a83dae8d None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:29:54.784875 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-9a382ff8-86c9-4175-8c6d-52e1a83dae8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:29:54.803252 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaca551-d36a-4bed-879c-dc62447eabaf None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:54.845778 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:54.845933 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ea662-30e7-4fe4-87f0-13f5937c6832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:54.847211 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ea662-30e7-4fe4-87f0-13f5937c6832 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:54.847789 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ea662-30e7-4fe4-87f0-13f5937c6832 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:54.848082 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ea662-30e7-4fe4-87f0-13f5937c6832 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:54.851030 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 37/77] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:29:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:29:54.851827 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a382ff8-86c9-4175-8c6d-52e1a83dae8d None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:54.922779 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:54.933530 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ea662-30e7-4fe4-87f0-13f5937c6832 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:54.935065 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ea662-30e7-4fe4-87f0-13f5937c6832 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:54.979870 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a382ff8-86c9-4175-8c6d-52e1a83dae8d None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:55.020234 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75d0561-4e30-4dd8-a74e-a9f82f4eb9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:55.020857 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d0561-4e30-4dd8-a74e-a9f82f4eb9f1 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:55.021142 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d0561-4e30-4dd8-a74e-a9f82f4eb9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:55.022130 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d0561-4e30-4dd8-a74e-a9f82f4eb9f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:55.022717 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75d0561-4e30-4dd8-a74e-a9f82f4eb9f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:29:55.023066 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75d0561-4e30-4dd8-a74e-a9f82f4eb9f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:55.024656 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75d0561-4e30-4dd8-a74e-a9f82f4eb9f1 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:55.034156 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 38/78] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:29:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:29:55.034952 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75d0561-4e30-4dd8-a74e-a9f82f4eb9f1 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:55.059760 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 41/79] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:29:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 12:29:55.059760 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ea662-30e7-4fe4-87f0-13f5937c6832 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:55.100943 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:55.147456 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75d0561-4e30-4dd8-a74e-a9f82f4eb9f1 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:55.159525 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:55.169543 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ea662-30e7-4fe4-87f0-13f5937c6832 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:55.190299 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5022c0-f496-4791-b2be-46e1a3e57094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:55.191239 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5022c0-f496-4791-b2be-46e1a3e57094 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:55.191708 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5022c0-f496-4791-b2be-46e1a3e57094 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:55.192134 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5022c0-f496-4791-b2be-46e1a3e57094 None admin] PATH_INFO: `/v3/users/b8101b669ca44866a11705e7d0f3b601` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:55.194383 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5022c0-f496-4791-b2be-46e1a3e57094 None admin] RBAC: Authorizing `identity:get_user(user_id=b8101b669ca44866a11705e7d0f3b601)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:55.196800 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5022c0-f496-4791-b2be-46e1a3e57094 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:55.199179 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 39/80] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:29:55 2022] GET /identity/v3/users/b8101b669ca44866a11705e7d0f3b601 => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:29:55.200210 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5022c0-f496-4791-b2be-46e1a3e57094 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:55.206729 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d7844-e8db-4ed6-8b66-ed2678d3ae42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:55.207394 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d7844-e8db-4ed6-8b66-ed2678d3ae42 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:55.207675 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d7844-e8db-4ed6-8b66-ed2678d3ae42 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:55.207950 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d7844-e8db-4ed6-8b66-ed2678d3ae42 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:55.218977 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3d7844-e8db-4ed6-8b66-ed2678d3ae42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:29:55.223810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-fa3d7844-e8db-4ed6-8b66-ed2678d3ae42 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 12:29:55.223810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:29:55.223810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:29:55.223810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:29:55.223810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:29:55.223810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:29:55.223810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:29:55.223810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:29:55.223810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:29:55.223810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:29:55.223810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:29:55.223810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:29:55.223810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:29:55.223810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:29:55.223810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:29:55.223810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:29:55.225201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:29:55.225201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:29:55.225201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:29:55.225201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:29:55.225201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:29:55.225201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:29:55.225201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:29:55.225201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:29:55.225201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:29:55.225201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:29:55.225201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:29:55.225201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:29:55.225201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:29:55.225201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:29:55.225201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:29:55.225201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:29:55.225963 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:29:55.225963 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:29:55.225963 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:29:55.225963 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:29:55.225963 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:29:55.225963 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:29:55.225963 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:29:55.225963 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:29:55.225963 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:29:55.225963 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:29:55.225963 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 12:29:55.225963 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:29:55.225963 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 42/81] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:29:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:29:55.225963 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d7844-e8db-4ed6-8b66-ed2678d3ae42 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:55.310438 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5022c0-f496-4791-b2be-46e1a3e57094 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:55.329505 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d7844-e8db-4ed6-8b66-ed2678d3ae42 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:55.336027 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:55.343837 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287cdd1-9db5-4d14-8474-a77090fe3a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:55.344474 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287cdd1-9db5-4d14-8474-a77090fe3a18 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:55.344751 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287cdd1-9db5-4d14-8474-a77090fe3a18 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:55.345024 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287cdd1-9db5-4d14-8474-a77090fe3a18 None admin] PATH_INFO: `/v3/users/77a14102256040ab81672f6adeaa6c5a` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:55.346148 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287cdd1-9db5-4d14-8474-a77090fe3a18 None admin] RBAC: Authorizing `identity:get_user(user_id=77a14102256040ab81672f6adeaa6c5a)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:55.347620 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287cdd1-9db5-4d14-8474-a77090fe3a18 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:55.349392 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 40/82] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:29:55 2022] GET /identity/v3/users/77a14102256040ab81672f6adeaa6c5a => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:29:55.350128 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287cdd1-9db5-4d14-8474-a77090fe3a18 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:55.368582 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a6a12-4c89-4722-8f7b-27ad0c9119bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:55.369213 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a6a12-4c89-4722-8f7b-27ad0c9119bb None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:55.369494 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a6a12-4c89-4722-8f7b-27ad0c9119bb None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:55.369766 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a6a12-4c89-4722-8f7b-27ad0c9119bb None admin] PATH_INFO: `/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:55.370878 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16a6a12-4c89-4722-8f7b-27ad0c9119bb None admin] RBAC: Authorizing `identity:get_project(project_id=6ee4b811a8dc4bc39e6ed0d4275b1782)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:55.372338 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16a6a12-4c89-4722-8f7b-27ad0c9119bb None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:55.374307 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 43/83] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:29:55 2022] GET /identity/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:29:55.379072 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a6a12-4c89-4722-8f7b-27ad0c9119bb None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:55.397919 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:55.457685 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287cdd1-9db5-4d14-8474-a77090fe3a18 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:55.475332 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a6a12-4c89-4722-8f7b-27ad0c9119bb None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:55.491738 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b419623-9d89-4e2a-859f-819d7a69f698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:55.492404 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b419623-9d89-4e2a-859f-819d7a69f698 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:55.492682 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b419623-9d89-4e2a-859f-819d7a69f698 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:55.492950 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b419623-9d89-4e2a-859f-819d7a69f698 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:55.494040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b419623-9d89-4e2a-859f-819d7a69f698 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:55.495500 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b419623-9d89-4e2a-859f-819d7a69f698 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:55.497068 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 41/84] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:29:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:29:55.497820 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b419623-9d89-4e2a-859f-819d7a69f698 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:55.509023 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f6bce-72f2-442c-acce-30dafcb8fb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:55.509636 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f6bce-72f2-442c-acce-30dafcb8fb63 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:55.509915 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f6bce-72f2-442c-acce-30dafcb8fb63 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:55.510190 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f6bce-72f2-442c-acce-30dafcb8fb63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:55.569317 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:55.589215 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3f6bce-72f2-442c-acce-30dafcb8fb63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:55.599486 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3f6bce-72f2-442c-acce-30dafcb8fb63 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:55.607396 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b419623-9d89-4e2a-859f-819d7a69f698 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:55.611349 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:55.641072 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530085cb-101e-428f-94e8-49f21ea3a3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:55.641687 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530085cb-101e-428f-94e8-49f21ea3a3e3 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:55.641993 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530085cb-101e-428f-94e8-49f21ea3a3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:55.642266 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530085cb-101e-428f-94e8-49f21ea3a3e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:55.642929 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530085cb-101e-428f-94e8-49f21ea3a3e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:29:55.643242 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530085cb-101e-428f-94e8-49f21ea3a3e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:55.644644 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530085cb-101e-428f-94e8-49f21ea3a3e3 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:55.653091 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 42/85] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 12:29:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:29:55.653817 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530085cb-101e-428f-94e8-49f21ea3a3e3 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:55.692868 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 44/86] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:29:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 12:29:55.693832 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f6bce-72f2-442c-acce-30dafcb8fb63 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:55.752357 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530085cb-101e-428f-94e8-49f21ea3a3e3 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:55.778235 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:55.786693 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c655f5e-3f6f-41bf-92de-389e7b048fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:55.788409 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c655f5e-3f6f-41bf-92de-389e7b048fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:55.788692 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c655f5e-3f6f-41bf-92de-389e7b048fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:55.788967 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c655f5e-3f6f-41bf-92de-389e7b048fe3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:55.792650 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f6bce-72f2-442c-acce-30dafcb8fb63 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:55.811419 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:55.825208 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fb7a7d-9b72-47d3-9855-b18f31b86a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:55.825812 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb7a7d-9b72-47d3-9855-b18f31b86a27 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:55.826099 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb7a7d-9b72-47d3-9855-b18f31b86a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:55.826380 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb7a7d-9b72-47d3-9855-b18f31b86a27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:55.827138 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fb7a7d-9b72-47d3-9855-b18f31b86a27 None admin] RBAC: Adding query filter params (role.id=47d5e2ea167241d5a27c8bbf4593eead, scope.project.id=6ee4b811a8dc4bc39e6ed0d4275b1782, user.id=b8101b669ca44866a11705e7d0f3b601) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:29:55.827490 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fb7a7d-9b72-47d3-9855-b18f31b86a27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:55.828938 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fb7a7d-9b72-47d3-9855-b18f31b86a27 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:55.835858 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 45/87] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:29:55 2022] GET /identity/v3/role_assignments?user.id=b8101b669ca44866a11705e7d0f3b601&scope.project.id=6ee4b811a8dc4bc39e6ed0d4275b1782&role.id=47d5e2ea167241d5a27c8bbf4593eead => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:29:55.836654 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fb7a7d-9b72-47d3-9855-b18f31b86a27 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:55.858949 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c655f5e-3f6f-41bf-92de-389e7b048fe3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:55.860101 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c655f5e-3f6f-41bf-92de-389e7b048fe3 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:55.972339 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fb7a7d-9b72-47d3-9855-b18f31b86a27 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:55.977493 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 43/88] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:29:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 12:29:55.978304 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c655f5e-3f6f-41bf-92de-389e7b048fe3 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:56.006009 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:56.019014 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d48a6f-9f4b-46ff-8149-49f0d4afcc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:56.020037 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d48a6f-9f4b-46ff-8149-49f0d4afcc42 None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:56.020484 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d48a6f-9f4b-46ff-8149-49f0d4afcc42 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:56.020928 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d48a6f-9f4b-46ff-8149-49f0d4afcc42 None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:56.022339 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d48a6f-9f4b-46ff-8149-49f0d4afcc42 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:56.024021 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d48a6f-9f4b-46ff-8149-49f0d4afcc42 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:56.080643 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c655f5e-3f6f-41bf-92de-389e7b048fe3 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:56.113282 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 46/89] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Oct 3 12:29:55 2022] POST /identity/v3/users => generated 289 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:29:56.123514 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae1e14-4120-4835-a1d6-f7f4179e65ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:56.123514 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae1e14-4120-4835-a1d6-f7f4179e65ce None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:56.123514 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae1e14-4120-4835-a1d6-f7f4179e65ce None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:56.123514 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae1e14-4120-4835-a1d6-f7f4179e65ce None admin] PATH_INFO: `/v3/users/b8101b669ca44866a11705e7d0f3b601` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:56.123801 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ae1e14-4120-4835-a1d6-f7f4179e65ce None admin] RBAC: Authorizing `identity:get_user(user_id=b8101b669ca44866a11705e7d0f3b601)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:56.123801 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ae1e14-4120-4835-a1d6-f7f4179e65ce None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:56.123924 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d48a6f-9f4b-46ff-8149-49f0d4afcc42 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:56.123987 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 44/90] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:29:55 2022] GET /identity/v3/users/b8101b669ca44866a11705e7d0f3b601 => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:29:56.123987 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae1e14-4120-4835-a1d6-f7f4179e65ce None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:56.200692 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:56.266496 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:56.275567 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae1e14-4120-4835-a1d6-f7f4179e65ce None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:56.275766 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d48a6f-9f4b-46ff-8149-49f0d4afcc42 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:56.314980 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644176e2-3667-4ff0-8277-bacdbf7a0424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:56.315922 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644176e2-3667-4ff0-8277-bacdbf7a0424 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:56.316345 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644176e2-3667-4ff0-8277-bacdbf7a0424 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:56.316591 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5264d3-ebda-4130-b768-fc71a4cbc3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:56.316917 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644176e2-3667-4ff0-8277-bacdbf7a0424 None admin] PATH_INFO: `/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:56.317244 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5264d3-ebda-4130-b768-fc71a4cbc3ed None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:56.317529 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5264d3-ebda-4130-b768-fc71a4cbc3ed None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:56.317808 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5264d3-ebda-4130-b768-fc71a4cbc3ed None admin] PATH_INFO: `/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:56.319560 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644176e2-3667-4ff0-8277-bacdbf7a0424 None admin] RBAC: Authorizing `identity:get_project(project_id=6ee4b811a8dc4bc39e6ed0d4275b1782)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:56.320004 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5264d3-ebda-4130-b768-fc71a4cbc3ed None admin] RBAC: Authorizing `identity:get_project(project_id=6ee4b811a8dc4bc39e6ed0d4275b1782)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:56.321014 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644176e2-3667-4ff0-8277-bacdbf7a0424 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:56.321461 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5264d3-ebda-4130-b768-fc71a4cbc3ed None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:56.322826 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 45/91] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:29:56 2022] GET /identity/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:29:56.323225 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 47/92] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:29:56 2022] GET /identity/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:29:56.327950 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644176e2-3667-4ff0-8277-bacdbf7a0424 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:56.329290 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5264d3-ebda-4130-b768-fc71a4cbc3ed None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:56.457426 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644176e2-3667-4ff0-8277-bacdbf7a0424 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:56.461652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5264d3-ebda-4130-b768-fc71a4cbc3ed None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:56.472733 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:56.506590 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd45d239-ef44-494b-bd2a-871143b2abf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:56.507216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45d239-ef44-494b-bd2a-871143b2abf3 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:56.507459 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45d239-ef44-494b-bd2a-871143b2abf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:56.507731 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45d239-ef44-494b-bd2a-871143b2abf3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:56.520693 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe52e87-bfb4-4000-90eb-4dadce092e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:56.521938 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe52e87-bfb4-4000-90eb-4dadce092e78 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:56.522361 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe52e87-bfb4-4000-90eb-4dadce092e78 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:56.522596 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe52e87-bfb4-4000-90eb-4dadce092e78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:56.523376 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe52e87-bfb4-4000-90eb-4dadce092e78 None admin] RBAC: Adding query filter params (role.id=7933c45dcdd54ef0a98018ba8a8bddaa, scope.project.id=6ee4b811a8dc4bc39e6ed0d4275b1782, user.id=77a14102256040ab81672f6adeaa6c5a) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:29:56.523667 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe52e87-bfb4-4000-90eb-4dadce092e78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:56.525139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe52e87-bfb4-4000-90eb-4dadce092e78 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:56.533243 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 48/93] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:29:56 2022] GET /identity/v3/role_assignments?user.id=77a14102256040ab81672f6adeaa6c5a&scope.project.id=6ee4b811a8dc4bc39e6ed0d4275b1782&role.id=7933c45dcdd54ef0a98018ba8a8bddaa => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:29:56.533909 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe52e87-bfb4-4000-90eb-4dadce092e78 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:56.558801 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:56.620340 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd45d239-ef44-494b-bd2a-871143b2abf3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:56.622950 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd45d239-ef44-494b-bd2a-871143b2abf3 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:56.663214 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe52e87-bfb4-4000-90eb-4dadce092e78 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:56.719808 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ba4c9c-b2b2-471a-a67c-d719b4da7de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:56.720305 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ba4c9c-b2b2-471a-a67c-d719b4da7de9 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:56.720588 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ba4c9c-b2b2-471a-a67c-d719b4da7de9 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:56.720862 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ba4c9c-b2b2-471a-a67c-d719b4da7de9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:56.721602 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ba4c9c-b2b2-471a-a67c-d719b4da7de9 None admin] RBAC: Adding query filter params (role.id=73b935a1ebc448899bbfc2381168d1cd, scope.project.id=6ee4b811a8dc4bc39e6ed0d4275b1782, user.id=b8101b669ca44866a11705e7d0f3b601) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:29:56.721911 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ba4c9c-b2b2-471a-a67c-d719b4da7de9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:56.723425 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ba4c9c-b2b2-471a-a67c-d719b4da7de9 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:56.739020 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 49/94] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:29:56 2022] GET /identity/v3/role_assignments?user.id=b8101b669ca44866a11705e7d0f3b601&scope.project.id=6ee4b811a8dc4bc39e6ed0d4275b1782&role.id=73b935a1ebc448899bbfc2381168d1cd => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:29:56.763791 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 46/95] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:29:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 12:29:56.769543 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ba4c9c-b2b2-471a-a67c-d719b4da7de9 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:56.811663 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:56.896820 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ba4c9c-b2b2-471a-a67c-d719b4da7de9 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:56.941234 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e9cc7b-3449-4225-ae76-42b7cf73f213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:56.942151 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e9cc7b-3449-4225-ae76-42b7cf73f213 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:56.942572 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e9cc7b-3449-4225-ae76-42b7cf73f213 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:56.943020 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e9cc7b-3449-4225-ae76-42b7cf73f213 None admin] PATH_INFO: `/v3/projects/456843bc98b345ffa5b05a5a92e99867` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:56.945044 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e9cc7b-3449-4225-ae76-42b7cf73f213 None admin] RBAC: Authorizing `identity:get_project(project_id=456843bc98b345ffa5b05a5a92e99867)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:56.946734 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e9cc7b-3449-4225-ae76-42b7cf73f213 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:56.948637 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 50/96] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:29:56 2022] GET /identity/v3/projects/456843bc98b345ffa5b05a5a92e99867 => generated 312 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:29:56.954916 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd45d239-ef44-494b-bd2a-871143b2abf3 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:57.048342 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:57.091459 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd45d239-ef44-494b-bd2a-871143b2abf3 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:57.126316 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba1e26-6ad7-4662-86e9-99e2006a1026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:57.126938 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba1e26-6ad7-4662-86e9-99e2006a1026 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:57.127220 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba1e26-6ad7-4662-86e9-99e2006a1026 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:57.127526 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba1e26-6ad7-4662-86e9-99e2006a1026 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:57.128256 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba1e26-6ad7-4662-86e9-99e2006a1026 None admin] RBAC: Adding query filter params (role.id=47d5e2ea167241d5a27c8bbf4593eead, scope.project.id=456843bc98b345ffa5b05a5a92e99867, user.id=b8101b669ca44866a11705e7d0f3b601) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:29:57.128570 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba1e26-6ad7-4662-86e9-99e2006a1026 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:57.130012 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba1e26-6ad7-4662-86e9-99e2006a1026 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:57.136591 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 47/97] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:29:56 2022] GET /identity/v3/role_assignments?user.id=b8101b669ca44866a11705e7d0f3b601&scope.project.id=456843bc98b345ffa5b05a5a92e99867&role.id=47d5e2ea167241d5a27c8bbf4593eead => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:29:58.528959 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd04bf4-cb07-4c54-b082-b21ebaf6e29e None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:58.529432 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd04bf4-cb07-4c54-b082-b21ebaf6e29e None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:58.529432 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd04bf4-cb07-4c54-b082-b21ebaf6e29e None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:58.531011 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 51/98] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:29:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:29:58.536391 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9efe366-b0fa-47df-8c8e-4c5dece887ca None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:58.536797 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9efe366-b0fa-47df-8c8e-4c5dece887ca None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:58.537039 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9efe366-b0fa-47df-8c8e-4c5dece887ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:58.560384 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-e9efe366-b0fa-47df-8c8e-4c5dece887ca None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:29:58.561606 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-e9efe366-b0fa-47df-8c8e-4c5dece887ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:29:58.652834 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 48/99] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:29:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:29:59.179291 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd04bf4-cb07-4c54-b082-b21ebaf6e29e None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:59.300937 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd04bf4-cb07-4c54-b082-b21ebaf6e29e None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:59.306698 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:59.347644 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d780d605-1a91-4aef-ba20-8721c6e776bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:59.348308 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780d605-1a91-4aef-ba20-8721c6e776bb None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:59.348677 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780d605-1a91-4aef-ba20-8721c6e776bb None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:59.348990 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780d605-1a91-4aef-ba20-8721c6e776bb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:59.363960 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d780d605-1a91-4aef-ba20-8721c6e776bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:29:59.369560 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-d780d605-1a91-4aef-ba20-8721c6e776bb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 12:29:59.369560 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:29:59.369560 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:29:59.369560 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:29:59.369560 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:29:59.369560 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:29:59.369560 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:29:59.369560 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:29:59.369560 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:29:59.369560 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:29:59.369560 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:29:59.369560 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:29:59.369560 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:29:59.369560 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:29:59.369560 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:29:59.369560 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:29:59.370390 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:29:59.370390 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:29:59.370390 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:29:59.370390 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:29:59.370390 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:29:59.370390 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:29:59.370390 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:29:59.370390 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:29:59.370390 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:29:59.370390 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:29:59.370390 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:29:59.370390 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:29:59.370390 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:29:59.370390 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:29:59.370390 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:29:59.370390 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:29:59.371567 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:29:59.371567 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:29:59.371567 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:29:59.371567 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:29:59.371567 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:29:59.371567 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:29:59.371567 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:29:59.371567 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:29:59.371567 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:29:59.371567 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:29:59.371567 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 12:29:59.371567 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:29:59.371567 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 52/100] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:29:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:29:59.378170 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9efe366-b0fa-47df-8c8e-4c5dece887ca None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:59.457729 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:59.504247 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9efe366-b0fa-47df-8c8e-4c5dece887ca None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:59.548069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca72f6-fb9d-44ea-bbd2-d18e2021c83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:59.548880 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca72f6-fb9d-44ea-bbd2-d18e2021c83a None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:59.549168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca72f6-fb9d-44ea-bbd2-d18e2021c83a None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:59.549438 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca72f6-fb9d-44ea-bbd2-d18e2021c83a None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:59.550112 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ca72f6-fb9d-44ea-bbd2-d18e2021c83a None admin] RBAC: Adding query filter params (name=member) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:29:59.550415 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ca72f6-fb9d-44ea-bbd2-d18e2021c83a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:59.551817 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ca72f6-fb9d-44ea-bbd2-d18e2021c83a None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:59.561877 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 49/101] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 12:29:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:29:59.567752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d780d605-1a91-4aef-ba20-8721c6e776bb None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:59.718443 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d780d605-1a91-4aef-ba20-8721c6e776bb None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:59.765551 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6549f7b-8e7b-44ce-bdb4-7b38138f974c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:59.766222 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6549f7b-8e7b-44ce-bdb4-7b38138f974c None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:59.766510 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6549f7b-8e7b-44ce-bdb4-7b38138f974c None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:59.766791 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6549f7b-8e7b-44ce-bdb4-7b38138f974c None admin] PATH_INFO: `/v3/users/b8101b669ca44866a11705e7d0f3b601` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:59.768004 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6549f7b-8e7b-44ce-bdb4-7b38138f974c None admin] RBAC: Authorizing `identity:get_user(user_id=b8101b669ca44866a11705e7d0f3b601)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:59.769460 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6549f7b-8e7b-44ce-bdb4-7b38138f974c None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:59.771966 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 53/102] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:29:59 2022] GET /identity/v3/users/b8101b669ca44866a11705e7d0f3b601 => generated 275 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:29:59.777490 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca72f6-fb9d-44ea-bbd2-d18e2021c83a None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:29:59.921758 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca72f6-fb9d-44ea-bbd2-d18e2021c83a None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:29:59.941157 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:29:59.953982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee0088-d83c-4bb8-b81d-cb1babb2a64e None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:59.954271 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee0088-d83c-4bb8-b81d-cb1babb2a64e None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:59.954534 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee0088-d83c-4bb8-b81d-cb1babb2a64e None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:59.959429 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 54/103] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:29:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:29:59.960564 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8386fca-920b-4e2f-aac9-25a0688db55f None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:59.960851 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8386fca-920b-4e2f-aac9-25a0688db55f None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:59.961122 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8386fca-920b-4e2f-aac9-25a0688db55f None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:59.966175 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e5c675-dda3-4df7-8b6f-257c9fa5bc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:29:59.966822 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 55/104] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:29:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:29:59.966990 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e5c675-dda3-4df7-8b6f-257c9fa5bc97 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:59.967166 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e5c675-dda3-4df7-8b6f-257c9fa5bc97 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:59.967378 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e5c675-dda3-4df7-8b6f-257c9fa5bc97 None admin] PATH_INFO: `/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:59.968587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e5c675-dda3-4df7-8b6f-257c9fa5bc97 None admin] RBAC: Authorizing `identity:get_project(project_id=6ee4b811a8dc4bc39e6ed0d4275b1782)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:29:59.970105 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e5c675-dda3-4df7-8b6f-257c9fa5bc97 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:29:59.970385 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9a702-2655-4d66-af05-811f95361cfc None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:59.970684 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9a702-2655-4d66-af05-811f95361cfc None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:59.970966 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9a702-2655-4d66-af05-811f95361cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:59.971913 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 50/105] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:29:59 2022] GET /identity/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:29:59.976818 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb2377-e7e3-4cc7-be18-ba0c4a7647d2 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:29:59.977109 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb2377-e7e3-4cc7-be18-ba0c4a7647d2 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:29:59.977393 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb2377-e7e3-4cc7-be18-ba0c4a7647d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:29:59.995235 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-90c9a702-2655-4d66-af05-811f95361cfc None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:29:59.996455 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-90c9a702-2655-4d66-af05-811f95361cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:00.006437 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-accb2377-e7e3-4cc7-be18-ba0c4a7647d2 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:00.007770 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-accb2377-e7e3-4cc7-be18-ba0c4a7647d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:00.053793 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:00.101946 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 51/106] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:29:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 12:30:00.103277 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb2377-e7e3-4cc7-be18-ba0c4a7647d2 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:00.104990 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 56/107] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:29:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:30:00.106247 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30e0b5-d461-4112-9532-3d60800f61ba None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:00.106542 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30e0b5-d461-4112-9532-3d60800f61ba None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:00.106814 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30e0b5-d461-4112-9532-3d60800f61ba None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:00.112998 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 57/108] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:00.114714 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f165e8a-b8bf-46f8-a69e-cdb74d0a2083 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:00.115353 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f165e8a-b8bf-46f8-a69e-cdb74d0a2083 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:00.115848 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f165e8a-b8bf-46f8-a69e-cdb74d0a2083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:00.135977 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-8f165e8a-b8bf-46f8-a69e-cdb74d0a2083 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:00.137676 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-8f165e8a-b8bf-46f8-a69e-cdb74d0a2083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:00.210508 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 58/109] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 12:30:00.215559 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb2377-e7e3-4cc7-be18-ba0c4a7647d2 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:00.234965 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:00.254096 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab84a9e-489c-4d3a-b862-99bc9a527518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:00.254833 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab84a9e-489c-4d3a-b862-99bc9a527518 None admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:00.255202 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab84a9e-489c-4d3a-b862-99bc9a527518 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:00.255551 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab84a9e-489c-4d3a-b862-99bc9a527518 None admin] PATH_INFO: `/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782/users/b8101b669ca44866a11705e7d0f3b601/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:00.257733 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab84a9e-489c-4d3a-b862-99bc9a527518 None admin] RBAC: Authorizing `identity:create_grant(project_id=6ee4b811a8dc4bc39e6ed0d4275b1782, user_id=b8101b669ca44866a11705e7d0f3b601, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:00.259230 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab84a9e-489c-4d3a-b862-99bc9a527518 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:00.280425 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc841b8-e2bc-430c-82d3-4bab4e6da258 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:00.281085 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc841b8-e2bc-430c-82d3-4bab4e6da258 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:00.281563 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc841b8-e2bc-430c-82d3-4bab4e6da258 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:00.287782 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 59/110] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:00.293721 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f9ace-e273-4950-922c-bc42bb39f2f2 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:00.294327 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f9ace-e273-4950-922c-bc42bb39f2f2 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:00.294798 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f9ace-e273-4950-922c-bc42bb39f2f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:00.320375 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-184f9ace-e273-4950-922c-bc42bb39f2f2 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:00.321909 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-184f9ace-e273-4950-922c-bc42bb39f2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:00.342461 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 52/111] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 12:30:00 2022] PUT /identity/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782/users/b8101b669ca44866a11705e7d0f3b601/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:30:00.391308 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a7a3d9-8806-49f7-892b-9ec2050057f5 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:00.391661 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a7a3d9-8806-49f7-892b-9ec2050057f5 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:00.391901 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a7a3d9-8806-49f7-892b-9ec2050057f5 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:00.393774 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 53/112] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 12:30:00.399994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac564f38-4703-4930-9b71-49f94fab07b9 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:00.400284 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac564f38-4703-4930-9b71-49f94fab07b9 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:00.400614 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac564f38-4703-4930-9b71-49f94fab07b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:00.438687 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-ac564f38-4703-4930-9b71-49f94fab07b9 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:00.439981 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-ac564f38-4703-4930-9b71-49f94fab07b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:00.446793 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 60/113] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:30:00.539172 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184f9ace-e273-4950-922c-bc42bb39f2f2 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:00.550533 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 54/114] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 12:30:00.587164 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:00.612747 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac564f38-4703-4930-9b71-49f94fab07b9 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:00.711084 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184f9ace-e273-4950-922c-bc42bb39f2f2 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:00.725648 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac564f38-4703-4930-9b71-49f94fab07b9 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:00.733533 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:00.765499 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ab141-16d4-4595-a77d-e045992f5456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:00.766170 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ab141-16d4-4595-a77d-e045992f5456 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:00.766452 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ab141-16d4-4595-a77d-e045992f5456 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:00.766724 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ab141-16d4-4595-a77d-e045992f5456 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:00.770604 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc8c3f-a327-4582-b8d7-228f6d7291cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:00.771022 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc8c3f-a327-4582-b8d7-228f6d7291cb None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:00.771430 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc8c3f-a327-4582-b8d7-228f6d7291cb None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:00.771871 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc8c3f-a327-4582-b8d7-228f6d7291cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:00.784184 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639ab141-16d4-4595-a77d-e045992f5456 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:00.790227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-639ab141-16d4-4595-a77d-e045992f5456 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 12:30:00.790227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:00.790227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:00.790227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:00.790227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:00.790227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:00.790227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:00.790227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:00.790227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:00.790227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:00.790227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:00.790227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:00.790227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:00.790227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:00.790227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:00.790227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:00.791040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:00.791040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:00.791040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:00.791040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:00.791040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:00.791040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:00.791040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:00.791040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:00.791040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:00.791040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:00.791040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:00.791040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:00.791040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:00.791040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:00.791040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:00.791040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:00.791933 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:00.791933 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:00.791933 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:00.791933 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:00.791933 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:00.791933 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:00.791933 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:00.791933 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:00.791933 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:00.791933 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:00.791933 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 12:30:00.791933 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:30:00.794381 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 55/115] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 12:30:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:30:00.796365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ab141-16d4-4595-a77d-e045992f5456 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:00.810785 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdc8c3f-a327-4582-b8d7-228f6d7291cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:00.817772 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-1cdc8c3f-a327-4582-b8d7-228f6d7291cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:30:00.817772 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:00.817772 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:00.817772 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:00.817772 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:00.817772 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:00.817772 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:00.817772 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:00.817772 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:00.817772 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:00.817772 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:00.817772 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:00.817772 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:00.817772 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:00.817772 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:00.817772 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:00.818688 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:00.818688 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:00.818688 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:00.818688 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:00.818688 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:00.818688 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:00.818688 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:00.818688 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:00.818688 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:00.818688 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:00.818688 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:00.818688 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:00.818688 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:00.818688 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:00.818688 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:00.818688 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:00.819475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:00.819475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:00.819475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:00.819475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:00.819475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:00.819475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:00.819475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:00.819475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:00.819475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:00.819475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:00.819475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:30:00.819475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:30:00.822402 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 61/116] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 12:30:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:30:00.823712 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc8c3f-a327-4582-b8d7-228f6d7291cb None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:00.903788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ab141-16d4-4595-a77d-e045992f5456 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:00.922434 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:00.941710 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60083753-4f68-499a-a7ea-bdd19c65bd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:00.942015 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc8c3f-a327-4582-b8d7-228f6d7291cb None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:00.942361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60083753-4f68-499a-a7ea-bdd19c65bd8f None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:00.942650 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60083753-4f68-499a-a7ea-bdd19c65bd8f None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:00.942926 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60083753-4f68-499a-a7ea-bdd19c65bd8f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:00.944190 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60083753-4f68-499a-a7ea-bdd19c65bd8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:00.945641 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60083753-4f68-499a-a7ea-bdd19c65bd8f None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:00.947302 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 56/117] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:30:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:00.951975 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60083753-4f68-499a-a7ea-bdd19c65bd8f None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:00.985986 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d0cdce-92c7-4047-8f17-383c7f5a9541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:00.986597 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d0cdce-92c7-4047-8f17-383c7f5a9541 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:00.986878 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d0cdce-92c7-4047-8f17-383c7f5a9541 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:00.987157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d0cdce-92c7-4047-8f17-383c7f5a9541 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:00.987865 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d0cdce-92c7-4047-8f17-383c7f5a9541 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:00.988017 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:00.988165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d0cdce-92c7-4047-8f17-383c7f5a9541 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:00.989553 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d0cdce-92c7-4047-8f17-383c7f5a9541 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:00.998366 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 62/118] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:30:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:00.999176 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d0cdce-92c7-4047-8f17-383c7f5a9541 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:01.065184 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60083753-4f68-499a-a7ea-bdd19c65bd8f None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:01.103805 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917bccb-dd01-4223-a3ad-2079d5b228ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:01.104448 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917bccb-dd01-4223-a3ad-2079d5b228ed None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:01.104729 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917bccb-dd01-4223-a3ad-2079d5b228ed None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:01.105000 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917bccb-dd01-4223-a3ad-2079d5b228ed None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:01.109957 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d0cdce-92c7-4047-8f17-383c7f5a9541 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:01.116637 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9917bccb-dd01-4223-a3ad-2079d5b228ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:01.121446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-9917bccb-dd01-4223-a3ad-2079d5b228ed None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 12:30:01.121446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:01.121446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:01.121446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:01.121446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:01.121446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:01.121446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:01.121446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:01.121446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:01.121446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:01.121446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:01.121446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:01.121446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:01.121446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:01.121446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:01.121446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:01.122293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:01.122293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:01.122293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:01.122293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:01.122293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:01.122293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:01.122293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:01.122293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:01.122293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:01.122293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:01.122293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:01.122293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:01.122293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:01.122293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:01.122293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:01.122293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:01.123040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:01.123040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:01.123040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:01.123040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:01.123040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:01.123040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:01.123040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:01.123040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:01.123040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:01.123040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:01.123040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 12:30:01.123040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:30:01.123040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 57/119] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:30:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:30:01.124245 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917bccb-dd01-4223-a3ad-2079d5b228ed None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:01.151691 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb0ef6-2cb7-48a5-8e0a-3069360ec842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:01.152312 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb0ef6-2cb7-48a5-8e0a-3069360ec842 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:01.152597 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb0ef6-2cb7-48a5-8e0a-3069360ec842 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:01.152889 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb0ef6-2cb7-48a5-8e0a-3069360ec842 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:01.153539 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befb0ef6-2cb7-48a5-8e0a-3069360ec842 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:01.153838 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befb0ef6-2cb7-48a5-8e0a-3069360ec842 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:01.155227 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befb0ef6-2cb7-48a5-8e0a-3069360ec842 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:01.165378 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 63/120] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:30:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:01.166470 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb0ef6-2cb7-48a5-8e0a-3069360ec842 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:01.174880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:01.229040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:01.236214 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917bccb-dd01-4223-a3ad-2079d5b228ed None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:01.271959 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb0ef6-2cb7-48a5-8e0a-3069360ec842 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:01.278401 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34b753-74c6-46e3-95fe-63e1c00d8f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:01.279043 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34b753-74c6-46e3-95fe-63e1c00d8f5f None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:01.279341 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34b753-74c6-46e3-95fe-63e1c00d8f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:01.279638 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34b753-74c6-46e3-95fe-63e1c00d8f5f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:01.291671 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f34b753-74c6-46e3-95fe-63e1c00d8f5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:01.296518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-9f34b753-74c6-46e3-95fe-63e1c00d8f5f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 12:30:01.296518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:01.296518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:01.296518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:01.296518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:01.296518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:01.296518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:01.296518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:01.296518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:01.296518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:01.296518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:01.296518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:01.296518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:01.296518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:01.296518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:01.296518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:01.297610 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:01.297610 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:01.297610 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:01.297610 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:01.297610 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:01.297610 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:01.297610 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:01.297610 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:01.297610 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:01.297610 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:01.297610 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:01.297610 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:01.297610 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:01.297610 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:01.297610 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:01.297610 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:01.298374 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:01.298374 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:01.298374 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:01.298374 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:01.298374 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:01.298374 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:01.298374 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:01.298374 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:01.298374 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:01.298374 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:01.298374 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 12:30:01.298374 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:30:01.298374 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 58/121] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:30:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:30:01.298374 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34b753-74c6-46e3-95fe-63e1c00d8f5f None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:01.316163 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba13214-2523-4cd3-b98e-79da0bdce805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:01.316709 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba13214-2523-4cd3-b98e-79da0bdce805 None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:01.316987 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba13214-2523-4cd3-b98e-79da0bdce805 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:01.317260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba13214-2523-4cd3-b98e-79da0bdce805 None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:01.317905 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba13214-2523-4cd3-b98e-79da0bdce805 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:01.319589 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba13214-2523-4cd3-b98e-79da0bdce805 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:01.402561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34b753-74c6-46e3-95fe-63e1c00d8f5f None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:01.405230 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 64/122] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Oct 3 12:30:01 2022] POST /identity/v3/projects => generated 302 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:30:01.406168 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba13214-2523-4cd3-b98e-79da0bdce805 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:01.450924 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acccd2ca-b13b-497e-a207-6887d58e2532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:01.452885 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acccd2ca-b13b-497e-a207-6887d58e2532 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:01.453318 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acccd2ca-b13b-497e-a207-6887d58e2532 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:01.453780 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acccd2ca-b13b-497e-a207-6887d58e2532 None admin] PATH_INFO: `/v3/users/b8101b669ca44866a11705e7d0f3b601` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:01.455221 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acccd2ca-b13b-497e-a207-6887d58e2532 None admin] RBAC: Authorizing `identity:get_user(user_id=b8101b669ca44866a11705e7d0f3b601)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:01.456910 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acccd2ca-b13b-497e-a207-6887d58e2532 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:01.461225 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 59/123] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:30:01 2022] GET /identity/v3/users/b8101b669ca44866a11705e7d0f3b601 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:01.462383 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acccd2ca-b13b-497e-a207-6887d58e2532 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:01.488423 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:01.496310 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:01.540425 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba13214-2523-4cd3-b98e-79da0bdce805 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:01.585445 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bcec7b-c6ec-42f7-898d-23c5949b89aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:01.586300 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bcec7b-c6ec-42f7-898d-23c5949b89aa None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:01.587596 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bcec7b-c6ec-42f7-898d-23c5949b89aa None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:01.588112 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bcec7b-c6ec-42f7-898d-23c5949b89aa None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:01.588840 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bcec7b-c6ec-42f7-898d-23c5949b89aa None admin] RBAC: Adding query filter params (name=reader) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:01.589232 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bcec7b-c6ec-42f7-898d-23c5949b89aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:01.591765 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bcec7b-c6ec-42f7-898d-23c5949b89aa None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:01.592408 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acccd2ca-b13b-497e-a207-6887d58e2532 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:01.604988 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 65/124] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 12:30:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:01.604988 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bcec7b-c6ec-42f7-898d-23c5949b89aa None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:01.634185 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c1ce4b-78a3-4028-8861-a33c29dc1a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:01.634976 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c1ce4b-78a3-4028-8861-a33c29dc1a52 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:01.635385 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c1ce4b-78a3-4028-8861-a33c29dc1a52 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:01.636016 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c1ce4b-78a3-4028-8861-a33c29dc1a52 None admin] PATH_INFO: `/v3/users/77a14102256040ab81672f6adeaa6c5a` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:01.637153 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c1ce4b-78a3-4028-8861-a33c29dc1a52 None admin] RBAC: Authorizing `identity:get_user(user_id=77a14102256040ab81672f6adeaa6c5a)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:01.638696 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c1ce4b-78a3-4028-8861-a33c29dc1a52 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:01.640555 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 60/125] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:30:01 2022] GET /identity/v3/users/77a14102256040ab81672f6adeaa6c5a => generated 247 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:01.642146 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c1ce4b-78a3-4028-8861-a33c29dc1a52 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:01.724106 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bcec7b-c6ec-42f7-898d-23c5949b89aa None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:01.744447 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:01.757271 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:01.760153 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbac8ca-647a-4a17-a356-8cb85606e670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:01.760771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbac8ca-647a-4a17-a356-8cb85606e670 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:01.761108 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbac8ca-647a-4a17-a356-8cb85606e670 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:01.761390 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbac8ca-647a-4a17-a356-8cb85606e670 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:01.762155 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbac8ca-647a-4a17-a356-8cb85606e670 None admin] RBAC: Adding query filter params (name=member) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:01.762464 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbac8ca-647a-4a17-a356-8cb85606e670 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:01.763957 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbac8ca-647a-4a17-a356-8cb85606e670 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:01.767616 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c1ce4b-78a3-4028-8861-a33c29dc1a52 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:01.773640 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 66/126] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 12:30:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:01.774388 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbac8ca-647a-4a17-a356-8cb85606e670 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:01.806125 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da139018-cdfa-412d-8ded-ddd437f4a4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:01.806772 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da139018-cdfa-412d-8ded-ddd437f4a4f8 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:01.807053 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da139018-cdfa-412d-8ded-ddd437f4a4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:01.807346 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da139018-cdfa-412d-8ded-ddd437f4a4f8 None admin] PATH_INFO: `/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:01.808560 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da139018-cdfa-412d-8ded-ddd437f4a4f8 None admin] RBAC: Authorizing `identity:get_project(project_id=6ee4b811a8dc4bc39e6ed0d4275b1782)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:01.810039 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da139018-cdfa-412d-8ded-ddd437f4a4f8 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:01.811789 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 61/127] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:30:01 2022] GET /identity/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:01.812716 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da139018-cdfa-412d-8ded-ddd437f4a4f8 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:01.901985 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbac8ca-647a-4a17-a356-8cb85606e670 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:01.937043 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da139018-cdfa-412d-8ded-ddd437f4a4f8 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:01.941943 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0c8f2e-4748-4305-90c0-8202cbbb7e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:01.942588 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c8f2e-4748-4305-90c0-8202cbbb7e38 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:01.942926 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c8f2e-4748-4305-90c0-8202cbbb7e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:01.943838 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c8f2e-4748-4305-90c0-8202cbbb7e38 None admin] PATH_INFO: `/v3/users/5c053331f4ae4779912272eadee0969a` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:01.958131 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0c8f2e-4748-4305-90c0-8202cbbb7e38 None admin] RBAC: Authorizing `identity:get_user(user_id=5c053331f4ae4779912272eadee0969a)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:01.960252 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0c8f2e-4748-4305-90c0-8202cbbb7e38 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:01.963442 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 67/128] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:30:01 2022] GET /identity/v3/users/5c053331f4ae4779912272eadee0969a => generated 289 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:01.964155 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0c8f2e-4748-4305-90c0-8202cbbb7e38 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:01.981346 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877ef956-3a6f-49aa-80ff-dd1f3d631bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:01.982250 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ef956-3a6f-49aa-80ff-dd1f3d631bef None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:01.982671 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ef956-3a6f-49aa-80ff-dd1f3d631bef None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:01.983395 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ef956-3a6f-49aa-80ff-dd1f3d631bef None admin] PATH_INFO: `/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:01.985041 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877ef956-3a6f-49aa-80ff-dd1f3d631bef None admin] RBAC: Authorizing `identity:get_project(project_id=6ee4b811a8dc4bc39e6ed0d4275b1782)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:01.986618 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877ef956-3a6f-49aa-80ff-dd1f3d631bef None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:01.989233 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 62/129] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:30:01 2022] GET /identity/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:01.990781 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877ef956-3a6f-49aa-80ff-dd1f3d631bef None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:02.010601 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:02.025099 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:02.070914 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0c8f2e-4748-4305-90c0-8202cbbb7e38 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:02.105739 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877ef956-3a6f-49aa-80ff-dd1f3d631bef None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:02.109745 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f7aac2-417a-4107-96d4-1ecb9d9739b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:02.110377 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f7aac2-417a-4107-96d4-1ecb9d9739b0 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:02.110656 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f7aac2-417a-4107-96d4-1ecb9d9739b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:02.110985 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f7aac2-417a-4107-96d4-1ecb9d9739b0 None admin] PATH_INFO: `/v3/users/b8101b669ca44866a11705e7d0f3b601` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:02.112208 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f7aac2-417a-4107-96d4-1ecb9d9739b0 None admin] RBAC: Authorizing `identity:get_user(user_id=b8101b669ca44866a11705e7d0f3b601)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:02.113659 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f7aac2-417a-4107-96d4-1ecb9d9739b0 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:02.116498 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 68/130] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:30:01 2022] GET /identity/v3/users/b8101b669ca44866a11705e7d0f3b601 => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:02.117553 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f7aac2-417a-4107-96d4-1ecb9d9739b0 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:02.143730 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4719b95-cd6c-4821-b8c2-b3a984e7a213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:02.144781 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4719b95-cd6c-4821-b8c2-b3a984e7a213 None admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:02.145552 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4719b95-cd6c-4821-b8c2-b3a984e7a213 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:02.146116 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4719b95-cd6c-4821-b8c2-b3a984e7a213 None admin] PATH_INFO: `/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782/users/b8101b669ca44866a11705e7d0f3b601/roles/73b935a1ebc448899bbfc2381168d1cd` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:02.148888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4719b95-cd6c-4821-b8c2-b3a984e7a213 None admin] RBAC: Authorizing `identity:create_grant(project_id=6ee4b811a8dc4bc39e6ed0d4275b1782, user_id=b8101b669ca44866a11705e7d0f3b601, role_id=73b935a1ebc448899bbfc2381168d1cd)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:02.150555 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4719b95-cd6c-4821-b8c2-b3a984e7a213 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:02.226193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f7aac2-417a-4107-96d4-1ecb9d9739b0 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:02.230111 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 63/131] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 12:30:01 2022] PUT /identity/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782/users/b8101b669ca44866a11705e7d0f3b601/roles/73b935a1ebc448899bbfc2381168d1cd => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:30:02.231169 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4719b95-cd6c-4821-b8c2-b3a984e7a213 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:02.237538 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:02.299850 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d16f249-9f63-4389-abc8-8ad862855bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:02.300589 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16f249-9f63-4389-abc8-8ad862855bcb None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:02.300871 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16f249-9f63-4389-abc8-8ad862855bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:02.301146 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16f249-9f63-4389-abc8-8ad862855bcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:02.373579 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:02.407236 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4719b95-cd6c-4821-b8c2-b3a984e7a213 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:02.445693 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d16f249-9f63-4389-abc8-8ad862855bcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:02.449699 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d16f249-9f63-4389-abc8-8ad862855bcb None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:02.455675 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36cc2b7-4477-41ef-b5d2-a4a21ed31e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:02.456359 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36cc2b7-4477-41ef-b5d2-a4a21ed31e3e None admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:02.456643 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36cc2b7-4477-41ef-b5d2-a4a21ed31e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:02.456917 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36cc2b7-4477-41ef-b5d2-a4a21ed31e3e None admin] PATH_INFO: `/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782/users/77a14102256040ab81672f6adeaa6c5a/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:02.464101 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36cc2b7-4477-41ef-b5d2-a4a21ed31e3e None admin] RBAC: Authorizing `identity:create_grant(project_id=6ee4b811a8dc4bc39e6ed0d4275b1782, user_id=77a14102256040ab81672f6adeaa6c5a, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:02.464101 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36cc2b7-4477-41ef-b5d2-a4a21ed31e3e None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:02.515278 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 64/132] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 12:30:02 2022] PUT /identity/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782/users/77a14102256040ab81672f6adeaa6c5a/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:30:02.516704 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36cc2b7-4477-41ef-b5d2-a4a21ed31e3e None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:02.649244 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36cc2b7-4477-41ef-b5d2-a4a21ed31e3e None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:02.686329 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 69/133] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:30:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 12:30:02.690902 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bad194-b2ee-43ab-bc47-56c8a62b7d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:02.691844 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bad194-b2ee-43ab-bc47-56c8a62b7d12 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:02.692332 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d16f249-9f63-4389-abc8-8ad862855bcb None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:02.692460 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bad194-b2ee-43ab-bc47-56c8a62b7d12 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:02.692980 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bad194-b2ee-43ab-bc47-56c8a62b7d12 None admin] PATH_INFO: `/v3/projects/456843bc98b345ffa5b05a5a92e99867` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:02.694420 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bad194-b2ee-43ab-bc47-56c8a62b7d12 None admin] RBAC: Authorizing `identity:get_project(project_id=456843bc98b345ffa5b05a5a92e99867)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:02.697606 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bad194-b2ee-43ab-bc47-56c8a62b7d12 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:02.702115 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 65/134] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:30:02 2022] GET /identity/v3/projects/456843bc98b345ffa5b05a5a92e99867 => generated 312 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:02.714235 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:02.717372 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bad194-b2ee-43ab-bc47-56c8a62b7d12 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:02.755847 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:02.869726 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d16f249-9f63-4389-abc8-8ad862855bcb None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:02.889453 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bad194-b2ee-43ab-bc47-56c8a62b7d12 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:02.911775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f947e45a-dafd-46a9-9283-c5406c753a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:02.913000 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f947e45a-dafd-46a9-9283-c5406c753a39 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:02.913284 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f947e45a-dafd-46a9-9283-c5406c753a39 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:02.913561 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f947e45a-dafd-46a9-9283-c5406c753a39 None admin] PATH_INFO: `/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:02.914986 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f947e45a-dafd-46a9-9283-c5406c753a39 None admin] RBAC: Authorizing `identity:get_project(project_id=6ee4b811a8dc4bc39e6ed0d4275b1782)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:02.916652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f947e45a-dafd-46a9-9283-c5406c753a39 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:02.918623 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 70/135] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:30:02 2022] GET /identity/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:02.924770 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f947e45a-dafd-46a9-9283-c5406c753a39 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:02.948917 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b24319-4aa6-4f33-8319-01e684c0292c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:02.949940 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b24319-4aa6-4f33-8319-01e684c0292c None admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:02.950399 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b24319-4aa6-4f33-8319-01e684c0292c None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:02.951196 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b24319-4aa6-4f33-8319-01e684c0292c None admin] PATH_INFO: `/v3/projects/456843bc98b345ffa5b05a5a92e99867/users/b8101b669ca44866a11705e7d0f3b601/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:02.956484 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b24319-4aa6-4f33-8319-01e684c0292c None admin] RBAC: Authorizing `identity:create_grant(project_id=456843bc98b345ffa5b05a5a92e99867, user_id=b8101b669ca44866a11705e7d0f3b601, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:02.961039 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b24319-4aa6-4f33-8319-01e684c0292c None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:03.034347 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 66/136] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 12:30:02 2022] PUT /identity/v3/projects/456843bc98b345ffa5b05a5a92e99867/users/b8101b669ca44866a11705e7d0f3b601/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:30:03.055798 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66eec0c-e25a-4257-934c-11a3fdfe314d None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:03.056458 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66eec0c-e25a-4257-934c-11a3fdfe314d None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:03.056934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66eec0c-e25a-4257-934c-11a3fdfe314d None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:03.058767 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 67/137] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 12:30:03.060898 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:03.067439 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684e737-b14e-43c6-b4ca-ae0b34281540 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:03.073003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684e737-b14e-43c6-b4ca-ae0b34281540 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:03.073003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684e737-b14e-43c6-b4ca-ae0b34281540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:03.090022 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f947e45a-dafd-46a9-9283-c5406c753a39 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:03.090445 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-1684e737-b14e-43c6-b4ca-ae0b34281540 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:03.091713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-1684e737-b14e-43c6-b4ca-ae0b34281540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:03.139260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab6536-1c05-416e-95bd-910fbc7d2465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:03.142366 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab6536-1c05-416e-95bd-910fbc7d2465 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:03.142661 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab6536-1c05-416e-95bd-910fbc7d2465 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:03.142940 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab6536-1c05-416e-95bd-910fbc7d2465 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:03.143711 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fab6536-1c05-416e-95bd-910fbc7d2465 None admin] RBAC: Adding query filter params (role.id=5c094fe41dc347cbbb61cb9342eee721, scope.project.id=6ee4b811a8dc4bc39e6ed0d4275b1782, user.id=5c053331f4ae4779912272eadee0969a) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:03.144028 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fab6536-1c05-416e-95bd-910fbc7d2465 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:03.145536 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fab6536-1c05-416e-95bd-910fbc7d2465 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:03.165486 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 71/138] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:30:02 2022] GET /identity/v3/role_assignments?user.id=5c053331f4ae4779912272eadee0969a&scope.project.id=6ee4b811a8dc4bc39e6ed0d4275b1782&role.id=5c094fe41dc347cbbb61cb9342eee721 => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:03.217216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 68/139] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 12:30:03.551007 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab6536-1c05-416e-95bd-910fbc7d2465 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:03.672198 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab6536-1c05-416e-95bd-910fbc7d2465 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:03.714158 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923b4d3-8213-4952-96e6-b84645154f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:03.715854 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923b4d3-8213-4952-96e6-b84645154f68 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:03.716343 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923b4d3-8213-4952-96e6-b84645154f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:03.716839 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923b4d3-8213-4952-96e6-b84645154f68 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:03.729409 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:03.733157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6923b4d3-8213-4952-96e6-b84645154f68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:03.738496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-6923b4d3-8213-4952-96e6-b84645154f68 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 12:30:03.738496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:03.738496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:03.738496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:03.738496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:03.738496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:03.738496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:03.738496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:03.738496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:03.738496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:03.738496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:03.738496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:03.738496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:03.738496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:03.738496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:03.738496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:03.739379 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:03.739379 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:03.739379 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:03.739379 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:03.739379 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:03.739379 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:03.739379 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:03.739379 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:03.739379 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:03.739379 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:03.739379 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:03.739379 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:03.739379 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:03.739379 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:03.739379 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:03.739379 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:03.740652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:03.740652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:03.740652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:03.740652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:03.740652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:03.740652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:03.740652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:03.740652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:03.740652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:03.740652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:03.740652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 12:30:03.740652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:30:03.740652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 72/140] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:30:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:30:03.747256 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1684e737-b14e-43c6-b4ca-ae0b34281540 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:03.763741 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:03.896487 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1684e737-b14e-43c6-b4ca-ae0b34281540 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:03.944873 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f31332-bcdd-4d1e-b131-0d2e76eacb44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:03.945501 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f31332-bcdd-4d1e-b131-0d2e76eacb44 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:03.945790 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f31332-bcdd-4d1e-b131-0d2e76eacb44 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:03.946068 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f31332-bcdd-4d1e-b131-0d2e76eacb44 None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:03.946748 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f31332-bcdd-4d1e-b131-0d2e76eacb44 None admin] RBAC: Adding query filter params (name=member) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:03.947055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f31332-bcdd-4d1e-b131-0d2e76eacb44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:03.948667 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f31332-bcdd-4d1e-b131-0d2e76eacb44 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:03.957467 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 69/141] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 12:30:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:03.964534 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923b4d3-8213-4952-96e6-b84645154f68 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:04.109018 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923b4d3-8213-4952-96e6-b84645154f68 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:04.145103 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e08d5d-0ad2-469d-8745-f1d982bfe922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:04.145750 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e08d5d-0ad2-469d-8745-f1d982bfe922 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:04.146029 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e08d5d-0ad2-469d-8745-f1d982bfe922 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:04.146354 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e08d5d-0ad2-469d-8745-f1d982bfe922 None admin] PATH_INFO: `/v3/users/b8101b669ca44866a11705e7d0f3b601` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:04.147637 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e08d5d-0ad2-469d-8745-f1d982bfe922 None admin] RBAC: Authorizing `identity:get_user(user_id=b8101b669ca44866a11705e7d0f3b601)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:04.149046 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e08d5d-0ad2-469d-8745-f1d982bfe922 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:04.151405 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 73/142] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:30:03 2022] GET /identity/v3/users/b8101b669ca44866a11705e7d0f3b601 => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:04.156157 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f31332-bcdd-4d1e-b131-0d2e76eacb44 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:04.273731 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:04.324717 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f31332-bcdd-4d1e-b131-0d2e76eacb44 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:04.325787 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f050c4e-2f93-4a8f-bf54-4b8810818c69 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:04.326072 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f050c4e-2f93-4a8f-bf54-4b8810818c69 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:04.326343 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f050c4e-2f93-4a8f-bf54-4b8810818c69 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:04.327864 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 74/143] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 12:30:04.341975 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e7727b-3da9-4e4b-a57e-3fa171e4de3d None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:04.343192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e7727b-3da9-4e4b-a57e-3fa171e4de3d None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:04.343850 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e7727b-3da9-4e4b-a57e-3fa171e4de3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:04.377137 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-95e7727b-3da9-4e4b-a57e-3fa171e4de3d None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:04.379384 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-95e7727b-3da9-4e4b-a57e-3fa171e4de3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:04.387883 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69f9b9-da0e-4c16-a21c-08056c8e8f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:04.388834 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69f9b9-da0e-4c16-a21c-08056c8e8f58 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:04.389342 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69f9b9-da0e-4c16-a21c-08056c8e8f58 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:04.389809 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69f9b9-da0e-4c16-a21c-08056c8e8f58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:04.516152 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:04.526334 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 75/144] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 12:30:04.530861 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d69f9b9-da0e-4c16-a21c-08056c8e8f58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:04.536643 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d69f9b9-da0e-4c16-a21c-08056c8e8f58 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:04.630583 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:04.656258 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 70/145] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:30:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 12:30:04.662390 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e7727b-3da9-4e4b-a57e-3fa171e4de3d None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:04.816989 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e7727b-3da9-4e4b-a57e-3fa171e4de3d None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:04.864034 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc08760-a180-481f-85d7-5f6fc55a27fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:04.864695 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc08760-a180-481f-85d7-5f6fc55a27fc None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:04.865010 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc08760-a180-481f-85d7-5f6fc55a27fc None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:04.865814 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc08760-a180-481f-85d7-5f6fc55a27fc None admin] PATH_INFO: `/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:04.866998 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc08760-a180-481f-85d7-5f6fc55a27fc None admin] RBAC: Authorizing `identity:get_project(project_id=6ee4b811a8dc4bc39e6ed0d4275b1782)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:04.868470 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc08760-a180-481f-85d7-5f6fc55a27fc None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:04.870311 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 76/146] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:30:04 2022] GET /identity/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:04.876349 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69f9b9-da0e-4c16-a21c-08056c8e8f58 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:04.934920 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc08760-a180-481f-85d7-5f6fc55a27fc None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:05.055582 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69f9b9-da0e-4c16-a21c-08056c8e8f58 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:05.062123 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:05.117729 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc62fe1f-58bc-469e-97dd-3e185f6df6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:05.119044 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc62fe1f-58bc-469e-97dd-3e185f6df6b0 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:05.119359 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc62fe1f-58bc-469e-97dd-3e185f6df6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:05.119622 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc62fe1f-58bc-469e-97dd-3e185f6df6b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:05.120407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc62fe1f-58bc-469e-97dd-3e185f6df6b0 None admin] RBAC: Adding query filter params (role.id=47d5e2ea167241d5a27c8bbf4593eead, scope.project.id=6ee4b811a8dc4bc39e6ed0d4275b1782, user.id=b8101b669ca44866a11705e7d0f3b601) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:05.120719 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc62fe1f-58bc-469e-97dd-3e185f6df6b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:05.122146 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc62fe1f-58bc-469e-97dd-3e185f6df6b0 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:05.129973 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc08760-a180-481f-85d7-5f6fc55a27fc None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:05.139762 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 71/147] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:30:04 2022] GET /identity/v3/role_assignments?user.id=b8101b669ca44866a11705e7d0f3b601&scope.project.id=6ee4b811a8dc4bc39e6ed0d4275b1782&role.id=47d5e2ea167241d5a27c8bbf4593eead => generated 608 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:05.179028 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c63aa6-c76b-49ba-bd22-bf1ba2ea0d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:05.179564 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c63aa6-c76b-49ba-bd22-bf1ba2ea0d90 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:05.179841 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c63aa6-c76b-49ba-bd22-bf1ba2ea0d90 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:05.180111 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c63aa6-c76b-49ba-bd22-bf1ba2ea0d90 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:05.181296 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c63aa6-c76b-49ba-bd22-bf1ba2ea0d90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:05.182972 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c63aa6-c76b-49ba-bd22-bf1ba2ea0d90 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:05.189466 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 77/148] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:30:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:30:05.191137 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc62fe1f-58bc-469e-97dd-3e185f6df6b0 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:05.233094 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:05.342931 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc62fe1f-58bc-469e-97dd-3e185f6df6b0 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:05.387906 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886cb1f2-5da1-4b98-a2ca-38c9add1061a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:05.388843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886cb1f2-5da1-4b98-a2ca-38c9add1061a None admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:05.389324 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886cb1f2-5da1-4b98-a2ca-38c9add1061a None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:05.389796 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886cb1f2-5da1-4b98-a2ca-38c9add1061a None admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:05.390646 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886cb1f2-5da1-4b98-a2ca-38c9add1061a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:05.392306 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886cb1f2-5da1-4b98-a2ca-38c9add1061a None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:05.424908 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528296dc-58ba-4eb1-9407-2e6fc0a55492 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:05.425516 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528296dc-58ba-4eb1-9407-2e6fc0a55492 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:05.425981 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528296dc-58ba-4eb1-9407-2e6fc0a55492 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:05.428273 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 78/149] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 12:30:05.435245 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf52c9-7e06-458e-9353-4974607c0c6e None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:05.435816 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf52c9-7e06-458e-9353-4974607c0c6e None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:05.436278 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf52c9-7e06-458e-9353-4974607c0c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:05.455566 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-8cdf52c9-7e06-458e-9353-4974607c0c6e None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:05.457032 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-8cdf52c9-7e06-458e-9353-4974607c0c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:05.502588 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 72/150] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Oct 3 12:30:05 2022] POST /identity/v3/users => generated 283 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:30:05.552672 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 79/151] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 12:30:05.873322 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886cb1f2-5da1-4b98-a2ca-38c9add1061a None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:05.986248 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:06.002514 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886cb1f2-5da1-4b98-a2ca-38c9add1061a None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:06.045317 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7180082e-c9e7-4b11-b7b4-66c13ff6d4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:06.045978 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180082e-c9e7-4b11-b7b4-66c13ff6d4ea None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:06.046255 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180082e-c9e7-4b11-b7b4-66c13ff6d4ea None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:06.046525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180082e-c9e7-4b11-b7b4-66c13ff6d4ea None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:06.060324 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7180082e-c9e7-4b11-b7b4-66c13ff6d4ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:06.065307 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-7180082e-c9e7-4b11-b7b4-66c13ff6d4ea None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 12:30:06.065307 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:06.065307 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:06.065307 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:06.065307 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:06.065307 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:06.065307 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:06.065307 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:06.065307 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:06.065307 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:06.065307 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:06.065307 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:06.065307 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:06.065307 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:06.065307 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:06.065307 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:06.066151 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:06.066151 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:06.066151 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:06.066151 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:06.066151 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:06.066151 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:06.066151 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:06.066151 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:06.066151 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:06.066151 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:06.066151 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:06.066151 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:06.066151 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:06.066151 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:06.066151 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:06.066151 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:06.066909 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:06.066909 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:06.066909 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:06.066909 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:06.066909 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:06.066909 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:06.066909 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:06.066909 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:06.066909 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:06.066909 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:06.066909 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 12:30:06.066909 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:30:06.069463 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 73/152] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 12:30:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:30:06.075129 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdf52c9-7e06-458e-9353-4974607c0c6e None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:06.075611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef2c02-8019-42b2-8cf8-c120ff252ddd None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:06.075885 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef2c02-8019-42b2-8cf8-c120ff252ddd None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:06.076155 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef2c02-8019-42b2-8cf8-c120ff252ddd None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:06.077632 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 74/153] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:06.082783 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe8fbd-7049-494e-8064-ae4a6d51d9f9 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:06.083069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe8fbd-7049-494e-8064-ae4a6d51d9f9 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:06.083369 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe8fbd-7049-494e-8064-ae4a6d51d9f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:06.104864 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-eafe8fbd-7049-494e-8064-ae4a6d51d9f9 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:06.106042 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-eafe8fbd-7049-494e-8064-ae4a6d51d9f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:06.185601 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:06.212260 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 75/154] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 12:30:06.242496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdf52c9-7e06-458e-9353-4974607c0c6e None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:06.291719 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e480bdf-ab81-411b-a377-c4de4096236c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:06.292346 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e480bdf-ab81-411b-a377-c4de4096236c None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:06.292627 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e480bdf-ab81-411b-a377-c4de4096236c None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:06.292903 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e480bdf-ab81-411b-a377-c4de4096236c None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:06.293584 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e480bdf-ab81-411b-a377-c4de4096236c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:06.293876 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e480bdf-ab81-411b-a377-c4de4096236c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:06.295262 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e480bdf-ab81-411b-a377-c4de4096236c None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:06.305202 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 80/155] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:30:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:06.310925 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe8fbd-7049-494e-8064-ae4a6d51d9f9 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:06.408210 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:06.436835 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe8fbd-7049-494e-8064-ae4a6d51d9f9 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:06.475429 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be7a9bf-fb4e-46a1-b8de-f893f8b89d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:06.476365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7a9bf-fb4e-46a1-b8de-f893f8b89d09 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:06.476914 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7a9bf-fb4e-46a1-b8de-f893f8b89d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:06.477351 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7a9bf-fb4e-46a1-b8de-f893f8b89d09 None admin] PATH_INFO: `/v3/users/b8101b669ca44866a11705e7d0f3b601` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:06.478672 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be7a9bf-fb4e-46a1-b8de-f893f8b89d09 None admin] RBAC: Authorizing `identity:get_user(user_id=b8101b669ca44866a11705e7d0f3b601)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:06.480318 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be7a9bf-fb4e-46a1-b8de-f893f8b89d09 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:06.482363 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 76/156] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:30:06 2022] GET /identity/v3/users/b8101b669ca44866a11705e7d0f3b601 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:06.488062 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e480bdf-ab81-411b-a377-c4de4096236c None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:06.538382 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be7a9bf-fb4e-46a1-b8de-f893f8b89d09 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:06.610594 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e480bdf-ab81-411b-a377-c4de4096236c None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:06.652731 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:06.663844 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d93001-de9c-4166-bc5a-55bebbb60c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:06.664471 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d93001-de9c-4166-bc5a-55bebbb60c9a None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:06.664747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d93001-de9c-4166-bc5a-55bebbb60c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:06.665018 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d93001-de9c-4166-bc5a-55bebbb60c9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:06.698528 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be7a9bf-fb4e-46a1-b8de-f893f8b89d09 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:06.744124 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb48e42d-164a-4cf9-a0aa-37ec6dd5f4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:06.744771 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48e42d-164a-4cf9-a0aa-37ec6dd5f4ea None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:06.745056 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48e42d-164a-4cf9-a0aa-37ec6dd5f4ea None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:06.745330 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48e42d-164a-4cf9-a0aa-37ec6dd5f4ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:06.758500 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb48e42d-164a-4cf9-a0aa-37ec6dd5f4ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:06.758663 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d93001-de9c-4166-bc5a-55bebbb60c9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:06.760193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d93001-de9c-4166-bc5a-55bebbb60c9a None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:06.764180 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-bb48e42d-164a-4cf9-a0aa-37ec6dd5f4ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:30:06.764180 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:06.764180 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:06.764180 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:06.764180 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:06.764180 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:06.764180 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:06.764180 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:06.764180 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:06.764180 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:06.764180 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:06.764180 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:06.764180 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:06.764180 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:06.764180 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:06.764180 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:06.765048 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:06.765048 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:06.765048 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:06.765048 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:06.765048 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:06.765048 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:06.765048 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:06.765048 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:06.765048 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:06.765048 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:06.765048 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:06.765048 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:06.765048 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:06.765048 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:06.765048 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:06.765048 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:06.765818 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:06.765818 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:06.765818 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:06.765818 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:06.765818 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:06.765818 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:06.765818 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:06.765818 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:06.765818 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:06.765818 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:06.765818 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:30:06.765818 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:30:06.765818 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 77/157] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 12:30:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:30:06.766510 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a889c76f-ba3c-41d5-8cf8-9dc4b6ce73bb None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:06.767281 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a889c76f-ba3c-41d5-8cf8-9dc4b6ce73bb None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:06.767281 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a889c76f-ba3c-41d5-8cf8-9dc4b6ce73bb None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:06.768419 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 78/158] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:06.773890 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28891052-71fe-4308-a172-d2b4b6375b89 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:06.774175 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28891052-71fe-4308-a172-d2b4b6375b89 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:06.774580 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28891052-71fe-4308-a172-d2b4b6375b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:06.783876 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:06.799806 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-28891052-71fe-4308-a172-d2b4b6375b89 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:06.800999 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-28891052-71fe-4308-a172-d2b4b6375b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:06.884580 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 79/159] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 12:30:06.884580 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28891052-71fe-4308-a172-d2b4b6375b89 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:06.898357 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 81/160] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:30:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 12:30:06.900168 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d200cbb-5f39-4ca4-8b90-b6af167896cc None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:06.900775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d200cbb-5f39-4ca4-8b90-b6af167896cc None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:06.901251 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d200cbb-5f39-4ca4-8b90-b6af167896cc None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:06.903068 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 82/161] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:06.908972 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d93ee-27a9-4ee6-8974-9f79f0bf8172 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:06.909548 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d93ee-27a9-4ee6-8974-9f79f0bf8172 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:06.910084 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d93ee-27a9-4ee6-8974-9f79f0bf8172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:06.941444 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-c31d93ee-27a9-4ee6-8974-9f79f0bf8172 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:06.942989 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-c31d93ee-27a9-4ee6-8974-9f79f0bf8172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:06.979765 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:06.990840 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28891052-71fe-4308-a172-d2b4b6375b89 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:07.025985 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e13829a-3ccf-4afa-b53a-7643b4bfbc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:07.026605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13829a-3ccf-4afa-b53a-7643b4bfbc04 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:07.026885 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13829a-3ccf-4afa-b53a-7643b4bfbc04 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:07.027159 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13829a-3ccf-4afa-b53a-7643b4bfbc04 None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:07.027975 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e13829a-3ccf-4afa-b53a-7643b4bfbc04 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:07.028281 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e13829a-3ccf-4afa-b53a-7643b4bfbc04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:07.029667 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e13829a-3ccf-4afa-b53a-7643b4bfbc04 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:07.031073 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:07.038603 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 80/162] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:30:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:07.039435 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e13829a-3ccf-4afa-b53a-7643b4bfbc04 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:07.065805 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 83/163] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:30:07.066479 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d93ee-27a9-4ee6-8974-9f79f0bf8172 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:07.207292 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e13829a-3ccf-4afa-b53a-7643b4bfbc04 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:07.272756 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39c780b-0e24-426d-8f9a-9de56242d692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:07.273421 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c780b-0e24-426d-8f9a-9de56242d692 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:07.274070 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c780b-0e24-426d-8f9a-9de56242d692 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:07.274070 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c780b-0e24-426d-8f9a-9de56242d692 None admin] PATH_INFO: `/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:07.279368 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39c780b-0e24-426d-8f9a-9de56242d692 None admin] RBAC: Authorizing `identity:get_project(project_id=6ee4b811a8dc4bc39e6ed0d4275b1782)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:07.279368 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39c780b-0e24-426d-8f9a-9de56242d692 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:07.280211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 81/164] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:30:07 2022] GET /identity/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782 => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:07.286090 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d93ee-27a9-4ee6-8974-9f79f0bf8172 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:07.286841 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39c780b-0e24-426d-8f9a-9de56242d692 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:07.333622 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc43c8-13ba-4f5c-b0b3-4ecea7e176ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:07.334939 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc43c8-13ba-4f5c-b0b3-4ecea7e176ce None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:07.335579 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc43c8-13ba-4f5c-b0b3-4ecea7e176ce None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:07.336086 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc43c8-13ba-4f5c-b0b3-4ecea7e176ce None admin] PATH_INFO: `/v3/users/77a14102256040ab81672f6adeaa6c5a` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:07.337915 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedc43c8-13ba-4f5c-b0b3-4ecea7e176ce None admin] RBAC: Authorizing `identity:get_user(user_id=77a14102256040ab81672f6adeaa6c5a)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:07.339776 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedc43c8-13ba-4f5c-b0b3-4ecea7e176ce None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:07.343276 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 84/165] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:30:07 2022] GET /identity/v3/users/77a14102256040ab81672f6adeaa6c5a => generated 247 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:07.351614 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc43c8-13ba-4f5c-b0b3-4ecea7e176ce None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:07.407182 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:07.418327 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:07.486738 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39c780b-0e24-426d-8f9a-9de56242d692 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:07.545556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc43c8-13ba-4f5c-b0b3-4ecea7e176ce None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:07.608050 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf731f3-75ae-4ba5-ad4f-72e1d41a25d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:07.608682 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf731f3-75ae-4ba5-ad4f-72e1d41a25d1 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:07.608866 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaee2b2-2585-41c8-8bde-794e3bdd278f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:07.608981 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf731f3-75ae-4ba5-ad4f-72e1d41a25d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:07.609230 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf731f3-75ae-4ba5-ad4f-72e1d41a25d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:07.609479 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaee2b2-2585-41c8-8bde-794e3bdd278f None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:07.609764 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaee2b2-2585-41c8-8bde-794e3bdd278f None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:07.610018 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf731f3-75ae-4ba5-ad4f-72e1d41a25d1 None admin] RBAC: Adding query filter params (role.id=73b935a1ebc448899bbfc2381168d1cd, scope.project.id=6ee4b811a8dc4bc39e6ed0d4275b1782, user.id=b8101b669ca44866a11705e7d0f3b601) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:07.610127 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaee2b2-2585-41c8-8bde-794e3bdd278f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:07.610335 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf731f3-75ae-4ba5-ad4f-72e1d41a25d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:07.611812 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf731f3-75ae-4ba5-ad4f-72e1d41a25d1 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:07.622471 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 82/166] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:30:07 2022] GET /identity/v3/role_assignments?user.id=b8101b669ca44866a11705e7d0f3b601&scope.project.id=6ee4b811a8dc4bc39e6ed0d4275b1782&role.id=73b935a1ebc448899bbfc2381168d1cd => generated 608 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:30:07.623301 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf731f3-75ae-4ba5-ad4f-72e1d41a25d1 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:07.786506 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaee2b2-2585-41c8-8bde-794e3bdd278f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:07.790884 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaee2b2-2585-41c8-8bde-794e3bdd278f None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:07.822677 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf731f3-75ae-4ba5-ad4f-72e1d41a25d1 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:07.847511 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:07.900309 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:07.924667 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42681eb-12cc-491b-bc20-fc485ee35b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:07.925632 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42681eb-12cc-491b-bc20-fc485ee35b82 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:07.926074 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42681eb-12cc-491b-bc20-fc485ee35b82 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:07.926505 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42681eb-12cc-491b-bc20-fc485ee35b82 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:07.938322 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 85/167] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:30:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 12:30:07.939343 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaee2b2-2585-41c8-8bde-794e3bdd278f None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:07.960338 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42681eb-12cc-491b-bc20-fc485ee35b82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:07.965845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-d42681eb-12cc-491b-bc20-fc485ee35b82 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 12:30:07.965845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:07.965845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:07.965845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:07.965845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:07.965845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:07.965845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:07.965845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:07.965845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:07.965845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:07.965845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:07.965845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:07.965845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:07.965845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:07.965845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:07.965845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:07.972269 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:07.972269 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:07.972269 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:07.972269 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:07.972269 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:07.972269 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:07.972269 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:07.972269 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:07.972269 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:07.972269 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:07.972269 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:07.972269 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:07.972269 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:07.972269 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:07.972269 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:07.972269 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:07.973092 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:07.973092 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:07.973092 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:07.973092 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:07.973092 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:07.973092 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:07.973092 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:07.973092 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:07.973092 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:07.973092 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:07.973092 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 12:30:07.973092 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:30:07.973092 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 83/168] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:30:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:30:07.973092 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42681eb-12cc-491b-bc20-fc485ee35b82 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:08.062097 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaee2b2-2585-41c8-8bde-794e3bdd278f None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:08.092923 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:08.102599 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be94e033-0e7c-4c97-8ade-7ba1ec0954fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:08.108297 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94e033-0e7c-4c97-8ade-7ba1ec0954fa None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:08.108297 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94e033-0e7c-4c97-8ade-7ba1ec0954fa None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:08.108297 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94e033-0e7c-4c97-8ade-7ba1ec0954fa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:08.127911 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be94e033-0e7c-4c97-8ade-7ba1ec0954fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:08.141454 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-be94e033-0e7c-4c97-8ade-7ba1ec0954fa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 12:30:08.141454 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:08.141454 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:08.141454 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:08.141454 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:08.141454 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:08.141454 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:08.141454 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:08.141454 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:08.141454 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:08.141454 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:08.141454 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:08.141454 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:08.141454 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:08.141454 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:08.141454 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:08.142444 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:08.142444 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:08.142444 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:08.142444 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:08.142444 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:08.142444 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:08.142444 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:08.142444 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:08.142444 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:08.142444 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:08.142444 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:08.142444 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:08.142444 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:08.142444 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:08.142444 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:08.142444 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:08.143306 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:08.143306 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:08.143306 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:08.143306 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:08.143306 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:08.143306 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:08.143306 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:08.143306 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:08.143306 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:08.143306 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:08.143306 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 12:30:08.143306 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:30:08.143932 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42681eb-12cc-491b-bc20-fc485ee35b82 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:08.144338 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 86/169] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:30:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:30:08.145200 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be94e033-0e7c-4c97-8ade-7ba1ec0954fa None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:08.202359 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1cfd6-b1e4-4caa-9e06-10dd4df35610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:08.202995 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1cfd6-b1e4-4caa-9e06-10dd4df35610 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:08.203274 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1cfd6-b1e4-4caa-9e06-10dd4df35610 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:08.203573 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1cfd6-b1e4-4caa-9e06-10dd4df35610 None admin] PATH_INFO: `/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:08.204701 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf1cfd6-b1e4-4caa-9e06-10dd4df35610 None admin] RBAC: Authorizing `identity:get_project(project_id=6ee4b811a8dc4bc39e6ed0d4275b1782)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:08.206698 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf1cfd6-b1e4-4caa-9e06-10dd4df35610 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:08.208460 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 84/170] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:30:07 2022] GET /identity/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782 => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:08.209911 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1cfd6-b1e4-4caa-9e06-10dd4df35610 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:08.280467 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:08.297511 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be94e033-0e7c-4c97-8ade-7ba1ec0954fa None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:08.328917 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1cfd6-b1e4-4caa-9e06-10dd4df35610 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:08.346811 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5972b6c-564f-49b6-a9c2-a783faa4c63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:08.349202 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5972b6c-564f-49b6-a9c2-a783faa4c63c None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:08.351222 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5972b6c-564f-49b6-a9c2-a783faa4c63c None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:08.352450 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5972b6c-564f-49b6-a9c2-a783faa4c63c None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:08.353483 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5972b6c-564f-49b6-a9c2-a783faa4c63c None admin] RBAC: Adding query filter params (name=member) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:08.355070 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5972b6c-564f-49b6-a9c2-a783faa4c63c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:08.357410 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5972b6c-564f-49b6-a9c2-a783faa4c63c None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:08.375103 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 87/171] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 12:30:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:08.377206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5972b6c-564f-49b6-a9c2-a783faa4c63c None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:08.390056 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e340b8-612c-4bf2-b249-54a8935fdb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:08.390958 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e340b8-612c-4bf2-b249-54a8935fdb0d None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:08.391460 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e340b8-612c-4bf2-b249-54a8935fdb0d None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:08.391927 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e340b8-612c-4bf2-b249-54a8935fdb0d None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:08.392835 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e340b8-612c-4bf2-b249-54a8935fdb0d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:08.393427 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e340b8-612c-4bf2-b249-54a8935fdb0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:08.395028 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e340b8-612c-4bf2-b249-54a8935fdb0d None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:08.409764 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 85/172] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 12:30:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:08.410895 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e340b8-612c-4bf2-b249-54a8935fdb0d None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:08.465342 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:08.550783 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e340b8-612c-4bf2-b249-54a8935fdb0d None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:08.572533 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:08.575191 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5972b6c-564f-49b6-a9c2-a783faa4c63c None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:08.597079 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2214dbcf-7012-4c40-b39a-8f1a90f4cb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:08.598046 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2214dbcf-7012-4c40-b39a-8f1a90f4cb40 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:08.598708 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2214dbcf-7012-4c40-b39a-8f1a90f4cb40 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:08.600725 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2214dbcf-7012-4c40-b39a-8f1a90f4cb40 None admin] PATH_INFO: `/v3/users/b8101b669ca44866a11705e7d0f3b601` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:08.602996 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2214dbcf-7012-4c40-b39a-8f1a90f4cb40 None admin] RBAC: Authorizing `identity:get_user(user_id=b8101b669ca44866a11705e7d0f3b601)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:08.606055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2214dbcf-7012-4c40-b39a-8f1a90f4cb40 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:08.617803 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 86/173] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:30:08 2022] GET /identity/v3/users/b8101b669ca44866a11705e7d0f3b601 => generated 275 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:08.617803 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2214dbcf-7012-4c40-b39a-8f1a90f4cb40 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:08.639439 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1127ef16-eb11-4c96-86ca-31036e15a180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:08.640650 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127ef16-eb11-4c96-86ca-31036e15a180 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:08.641128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127ef16-eb11-4c96-86ca-31036e15a180 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:08.641592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127ef16-eb11-4c96-86ca-31036e15a180 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:08.642525 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1127ef16-eb11-4c96-86ca-31036e15a180 None admin] RBAC: Adding query filter params (role.id=7933c45dcdd54ef0a98018ba8a8bddaa, scope.project.id=6ee4b811a8dc4bc39e6ed0d4275b1782, user.id=77a14102256040ab81672f6adeaa6c5a) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:08.643474 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1127ef16-eb11-4c96-86ca-31036e15a180 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:08.645568 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1127ef16-eb11-4c96-86ca-31036e15a180 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:08.663164 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 88/174] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:30:08 2022] GET /identity/v3/role_assignments?user.id=77a14102256040ab81672f6adeaa6c5a&scope.project.id=6ee4b811a8dc4bc39e6ed0d4275b1782&role.id=7933c45dcdd54ef0a98018ba8a8bddaa => generated 608 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:08.666147 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1127ef16-eb11-4c96-86ca-31036e15a180 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:08.770412 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2214dbcf-7012-4c40-b39a-8f1a90f4cb40 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:08.827353 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6af36-851e-46eb-ae78-14c5a2e7131b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:08.827353 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6af36-851e-46eb-ae78-14c5a2e7131b None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:08.827353 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6af36-851e-46eb-ae78-14c5a2e7131b None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:08.827353 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6af36-851e-46eb-ae78-14c5a2e7131b None admin] PATH_INFO: `/v3/users/5c053331f4ae4779912272eadee0969a` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:08.827755 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f6af36-851e-46eb-ae78-14c5a2e7131b None admin] RBAC: Authorizing `identity:get_user(user_id=5c053331f4ae4779912272eadee0969a)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:08.830041 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f6af36-851e-46eb-ae78-14c5a2e7131b None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:08.832775 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 87/175] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:30:08 2022] GET /identity/v3/users/5c053331f4ae4779912272eadee0969a => generated 289 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:08.844492 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6af36-851e-46eb-ae78-14c5a2e7131b None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:08.891406 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1127ef16-eb11-4c96-86ca-31036e15a180 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:08.896137 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:08.911702 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:08.952678 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df218b-014d-4fa1-b526-e773f49f727d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:08.953703 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df218b-014d-4fa1-b526-e773f49f727d None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:08.954177 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df218b-014d-4fa1-b526-e773f49f727d None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:08.954640 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df218b-014d-4fa1-b526-e773f49f727d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:08.963881 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6af36-851e-46eb-ae78-14c5a2e7131b None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:09.007340 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced5083e-8239-42a4-a72a-fa3f082d9ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:09.007340 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced5083e-8239-42a4-a72a-fa3f082d9ba4 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:09.007340 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced5083e-8239-42a4-a72a-fa3f082d9ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:09.007340 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced5083e-8239-42a4-a72a-fa3f082d9ba4 None admin] PATH_INFO: `/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:09.007735 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced5083e-8239-42a4-a72a-fa3f082d9ba4 None admin] RBAC: Authorizing `identity:get_project(project_id=6ee4b811a8dc4bc39e6ed0d4275b1782)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:09.007735 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced5083e-8239-42a4-a72a-fa3f082d9ba4 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:09.010076 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 88/176] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:30:08 2022] GET /identity/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:30:09.020450 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced5083e-8239-42a4-a72a-fa3f082d9ba4 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:09.063272 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7df218b-014d-4fa1-b526-e773f49f727d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:09.065150 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7df218b-014d-4fa1-b526-e773f49f727d None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:09.174783 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 89/177] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:30:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 12:30:09.181344 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df218b-014d-4fa1-b526-e773f49f727d None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:09.196127 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:09.200406 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced5083e-8239-42a4-a72a-fa3f082d9ba4 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:09.239755 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:09.257043 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfafb637-438e-4890-b64e-ed4863b07c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:09.257735 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfafb637-438e-4890-b64e-ed4863b07c1e None admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:09.258017 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfafb637-438e-4890-b64e-ed4863b07c1e None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:09.258290 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfafb637-438e-4890-b64e-ed4863b07c1e None admin] PATH_INFO: `/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782/users/5c053331f4ae4779912272eadee0969a/roles/5c094fe41dc347cbbb61cb9342eee721` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:09.261443 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfafb637-438e-4890-b64e-ed4863b07c1e None admin] RBAC: Authorizing `identity:create_grant(project_id=6ee4b811a8dc4bc39e6ed0d4275b1782, user_id=5c053331f4ae4779912272eadee0969a, role_id=5c094fe41dc347cbbb61cb9342eee721)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:09.262883 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfafb637-438e-4890-b64e-ed4863b07c1e None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:09.331365 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df218b-014d-4fa1-b526-e773f49f727d None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:09.343986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 89/178] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 12:30:09 2022] PUT /identity/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782/users/5c053331f4ae4779912272eadee0969a/roles/5c094fe41dc347cbbb61cb9342eee721 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:30:09.346014 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c2c26-32af-4d86-b934-c6fd5f9c9f80 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:09.346460 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c2c26-32af-4d86-b934-c6fd5f9c9f80 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:09.346920 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c2c26-32af-4d86-b934-c6fd5f9c9f80 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:09.348827 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 90/179] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:09.356514 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f3772-d8d6-4dba-8da2-b47af86c2feb None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:09.357002 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f3772-d8d6-4dba-8da2-b47af86c2feb None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:09.357437 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f3772-d8d6-4dba-8da2-b47af86c2feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:09.382341 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-682f3772-d8d6-4dba-8da2-b47af86c2feb None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:09.384063 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-682f3772-d8d6-4dba-8da2-b47af86c2feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:09.416274 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a110c18-f0f0-4552-a9c1-8f60358974b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:09.416930 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a110c18-f0f0-4552-a9c1-8f60358974b2 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:09.417275 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a110c18-f0f0-4552-a9c1-8f60358974b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:09.417622 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a110c18-f0f0-4552-a9c1-8f60358974b2 None admin] PATH_INFO: `/v3/projects/456843bc98b345ffa5b05a5a92e99867` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:09.418979 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a110c18-f0f0-4552-a9c1-8f60358974b2 None admin] RBAC: Authorizing `identity:get_project(project_id=456843bc98b345ffa5b05a5a92e99867)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:09.420709 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a110c18-f0f0-4552-a9c1-8f60358974b2 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:09.422556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 90/180] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:30:09 2022] GET /identity/v3/projects/456843bc98b345ffa5b05a5a92e99867 => generated 312 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:30:09.428658 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a110c18-f0f0-4552-a9c1-8f60358974b2 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:09.481474 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 91/181] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 12:30:09.544925 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6d448-31a3-4506-82f8-9505a0ef3a41 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:09.544925 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6d448-31a3-4506-82f8-9505a0ef3a41 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:09.545214 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6d448-31a3-4506-82f8-9505a0ef3a41 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:09.546786 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 92/182] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 12:30:09.552860 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a151a7-6b12-497f-a858-2168cd405579 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:09.553253 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a151a7-6b12-497f-a858-2168cd405579 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:09.553531 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a151a7-6b12-497f-a858-2168cd405579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:09.574135 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-97a151a7-6b12-497f-a858-2168cd405579 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:09.575373 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-97a151a7-6b12-497f-a858-2168cd405579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:09.591455 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:09.641859 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a110c18-f0f0-4552-a9c1-8f60358974b2 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:09.671412 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 93/183] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 12:30:09.686084 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04bc004-8007-49ef-92ca-23052543389b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:09.686710 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04bc004-8007-49ef-92ca-23052543389b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:09.687061 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04bc004-8007-49ef-92ca-23052543389b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:09.687443 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04bc004-8007-49ef-92ca-23052543389b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:09.688234 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04bc004-8007-49ef-92ca-23052543389b None admin] RBAC: Adding query filter params (role.id=47d5e2ea167241d5a27c8bbf4593eead, scope.project.id=456843bc98b345ffa5b05a5a92e99867, user.id=b8101b669ca44866a11705e7d0f3b601) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:09.688610 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04bc004-8007-49ef-92ca-23052543389b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:09.690188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04bc004-8007-49ef-92ca-23052543389b None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:09.696868 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 91/184] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:30:09 2022] GET /identity/v3/role_assignments?user.id=b8101b669ca44866a11705e7d0f3b601&scope.project.id=456843bc98b345ffa5b05a5a92e99867&role.id=47d5e2ea167241d5a27c8bbf4593eead => generated 608 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:09.814388 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a151a7-6b12-497f-a858-2168cd405579 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:09.863622 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:09.931882 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a151a7-6b12-497f-a858-2168cd405579 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:09.960185 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04bc004-8007-49ef-92ca-23052543389b None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:10.013332 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd12c7-4502-445b-8d82-76458d69084f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:10.013989 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd12c7-4502-445b-8d82-76458d69084f None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:10.014267 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd12c7-4502-445b-8d82-76458d69084f None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:10.014537 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd12c7-4502-445b-8d82-76458d69084f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:10.032860 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bd12c7-4502-445b-8d82-76458d69084f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:10.038783 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-a4bd12c7-4502-445b-8d82-76458d69084f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:30:10.038783 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:10.038783 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:10.038783 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:10.038783 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:10.038783 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:10.038783 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:10.038783 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:10.038783 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:10.038783 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:10.038783 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:10.038783 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:10.038783 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:10.038783 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:10.038783 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:10.038783 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:10.039763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:10.039763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:10.039763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:10.039763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:10.039763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:10.039763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:10.039763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:10.039763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:10.039763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:10.039763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:10.039763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:10.039763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:10.039763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:10.039763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:10.039763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:10.039763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:10.040587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:10.040587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:10.040587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:10.040587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:10.040587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:10.040587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:10.040587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:10.040587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:10.040587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:10.040587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:10.040587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:30:10.040587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:30:10.040587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 94/185] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 12:30:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:30:10.041277 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a76440-aed2-4272-a6e6-f8791acf43d5 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:10.041858 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a76440-aed2-4272-a6e6-f8791acf43d5 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:10.041858 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a76440-aed2-4272-a6e6-f8791acf43d5 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:10.043066 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 95/186] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:10.048496 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d940d72e-92e3-4dcf-93c0-d72f80ba620b None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:10.048780 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d940d72e-92e3-4dcf-93c0-d72f80ba620b None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:10.049076 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d940d72e-92e3-4dcf-93c0-d72f80ba620b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:10.070270 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-d940d72e-92e3-4dcf-93c0-d72f80ba620b None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:10.071958 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-d940d72e-92e3-4dcf-93c0-d72f80ba620b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:10.092528 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04bc004-8007-49ef-92ca-23052543389b None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:10.129922 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c0b37-8fdc-42f4-a259-570591dd34a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:10.132170 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c0b37-8fdc-42f4-a259-570591dd34a0 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:10.132666 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c0b37-8fdc-42f4-a259-570591dd34a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:10.133207 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c0b37-8fdc-42f4-a259-570591dd34a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:10.134650 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6c0b37-8fdc-42f4-a259-570591dd34a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:10.136542 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6c0b37-8fdc-42f4-a259-570591dd34a0 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:10.138493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 92/187] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:30:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:10.139570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c0b37-8fdc-42f4-a259-570591dd34a0 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:10.143964 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 96/188] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 12:30:10.145056 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d940d72e-92e3-4dcf-93c0-d72f80ba620b None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:10.158502 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:10.183639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:10.263463 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c0b37-8fdc-42f4-a259-570591dd34a0 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:10.278622 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d940d72e-92e3-4dcf-93c0-d72f80ba620b None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:10.303763 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad297a-be65-4fc0-8287-fb59f14089cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:10.304671 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad297a-be65-4fc0-8287-fb59f14089cd None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:10.305043 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad297a-be65-4fc0-8287-fb59f14089cd None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:10.305409 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad297a-be65-4fc0-8287-fb59f14089cd None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:10.306162 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ad297a-be65-4fc0-8287-fb59f14089cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:10.306560 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ad297a-be65-4fc0-8287-fb59f14089cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:10.308062 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ad297a-be65-4fc0-8287-fb59f14089cd None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:10.316654 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 93/189] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:30:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:10.322297 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad297a-be65-4fc0-8287-fb59f14089cd None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:10.324541 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e292e9-ffaa-4252-856c-d8dcd2a1592f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:10.325643 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e292e9-ffaa-4252-856c-d8dcd2a1592f None admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:10.326122 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e292e9-ffaa-4252-856c-d8dcd2a1592f None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:10.327552 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e292e9-ffaa-4252-856c-d8dcd2a1592f None admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:10.328423 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e292e9-ffaa-4252-856c-d8dcd2a1592f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:10.330042 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e292e9-ffaa-4252-856c-d8dcd2a1592f None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:10.395831 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:10.426301 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad297a-be65-4fc0-8287-fb59f14089cd None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:10.447376 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 97/190] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Oct 3 12:30:10 2022] POST /identity/v3/users => generated 297 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:30:10.451369 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e292e9-ffaa-4252-856c-d8dcd2a1592f None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:10.463941 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd3caf-cbb6-48cf-a7a9-5a0cef3635bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:10.464943 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd3caf-cbb6-48cf-a7a9-5a0cef3635bb None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:10.465906 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd3caf-cbb6-48cf-a7a9-5a0cef3635bb None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:10.466368 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd3caf-cbb6-48cf-a7a9-5a0cef3635bb None admin] PATH_INFO: `/v3/users/b17271447780411384122cb456a89cb0` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:10.486200 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cd3caf-cbb6-48cf-a7a9-5a0cef3635bb None admin] RBAC: Authorizing `identity:get_user(user_id=b17271447780411384122cb456a89cb0)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:10.488062 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cd3caf-cbb6-48cf-a7a9-5a0cef3635bb None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:10.490620 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 94/191] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:30:10 2022] GET /identity/v3/users/b17271447780411384122cb456a89cb0 => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:10.496686 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd3caf-cbb6-48cf-a7a9-5a0cef3635bb None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:10.585943 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:10.604495 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e292e9-ffaa-4252-856c-d8dcd2a1592f None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:10.622472 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd3caf-cbb6-48cf-a7a9-5a0cef3635bb None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:10.655848 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e8d711-a836-48e2-a323-8feef49e24f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:10.657152 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e8d711-a836-48e2-a323-8feef49e24f3 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:10.657152 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e8d711-a836-48e2-a323-8feef49e24f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:10.657152 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e8d711-a836-48e2-a323-8feef49e24f3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:10.664781 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:10.672279 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3008cd-55e9-44f3-8fe9-94a950176534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:10.674861 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3008cd-55e9-44f3-8fe9-94a950176534 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:10.674861 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3008cd-55e9-44f3-8fe9-94a950176534 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:10.674861 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3008cd-55e9-44f3-8fe9-94a950176534 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:10.675093 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e8d711-a836-48e2-a323-8feef49e24f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:10.681255 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-49e8d711-a836-48e2-a323-8feef49e24f3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 12:30:10.681255 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:10.681255 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:10.681255 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:10.681255 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:10.681255 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:10.681255 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:10.681255 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:10.681255 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:10.681255 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:10.681255 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:10.681255 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:10.681255 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:10.681255 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:10.681255 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:10.681255 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:10.682143 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:10.682143 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:10.682143 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:10.682143 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:10.682143 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:10.682143 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:10.682143 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:10.682143 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:10.682143 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:10.682143 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:10.682143 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:10.682143 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:10.682143 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:10.682143 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:10.682143 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:10.682143 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:10.682952 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:10.682952 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:10.682952 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:10.682952 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:10.682952 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:10.682952 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:10.682952 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:10.682952 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:10.682952 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:10.682952 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:10.682952 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 12:30:10.682952 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:30:10.682952 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 98/192] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 12:30:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:30:10.690281 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e8d711-a836-48e2-a323-8feef49e24f3 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:10.755259 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3008cd-55e9-44f3-8fe9-94a950176534 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:10.757864 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3008cd-55e9-44f3-8fe9-94a950176534 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:10.809295 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e8d711-a836-48e2-a323-8feef49e24f3 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:10.856160 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8ee8e-9de3-4df8-8ee6-0e3b337f95b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:10.856719 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8ee8e-9de3-4df8-8ee6-0e3b337f95b0 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:10.857005 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8ee8e-9de3-4df8-8ee6-0e3b337f95b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:10.857279 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8ee8e-9de3-4df8-8ee6-0e3b337f95b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:10.857958 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8ee8e-9de3-4df8-8ee6-0e3b337f95b0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:10.858267 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8ee8e-9de3-4df8-8ee6-0e3b337f95b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:10.859769 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8ee8e-9de3-4df8-8ee6-0e3b337f95b0 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:10.869989 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 99/193] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:30:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:30:10.876233 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8ee8e-9de3-4df8-8ee6-0e3b337f95b0 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:10.891015 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:10.894714 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 95/194] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:30:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 03 12:30:10.895545 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3008cd-55e9-44f3-8fe9-94a950176534 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:10.934559 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:11.012640 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8ee8e-9de3-4df8-8ee6-0e3b337f95b0 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:11.018911 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3008cd-55e9-44f3-8fe9-94a950176534 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:11.063973 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad3f77-2656-4589-aea2-190500f7fa3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:11.064621 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad3f77-2656-4589-aea2-190500f7fa3c None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:11.064899 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad3f77-2656-4589-aea2-190500f7fa3c None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:11.065169 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad3f77-2656-4589-aea2-190500f7fa3c None admin] PATH_INFO: `/v3/users/b17271447780411384122cb456a89cb0` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:11.066717 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ad3f77-2656-4589-aea2-190500f7fa3c None admin] RBAC: Authorizing `identity:get_user(user_id=b17271447780411384122cb456a89cb0)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:11.068919 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ad3f77-2656-4589-aea2-190500f7fa3c None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:11.070819 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 100/195] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:30:10 2022] GET /identity/v3/users/b17271447780411384122cb456a89cb0 => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:30:11.071144 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad500a8f-0458-4c53-b4db-c7c84103d81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:11.071944 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad500a8f-0458-4c53-b4db-c7c84103d81e None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:11.072231 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad500a8f-0458-4c53-b4db-c7c84103d81e None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:11.072507 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad500a8f-0458-4c53-b4db-c7c84103d81e None admin] PATH_INFO: `/v3/projects/c974a924a68947af9e8abc10c2614ae7` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:11.073676 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad500a8f-0458-4c53-b4db-c7c84103d81e None admin] RBAC: Authorizing `identity:get_project(project_id=c974a924a68947af9e8abc10c2614ae7)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:11.075284 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad500a8f-0458-4c53-b4db-c7c84103d81e None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:11.076072 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad3f77-2656-4589-aea2-190500f7fa3c None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:11.082824 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 96/196] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:30:10 2022] GET /identity/v3/projects/c974a924a68947af9e8abc10c2614ae7 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:11.083396 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad500a8f-0458-4c53-b4db-c7c84103d81e None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:11.175950 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:11.205079 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad3f77-2656-4589-aea2-190500f7fa3c None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:11.258380 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f046f1-8572-4d23-af72-d2174a80069f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:11.259008 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f046f1-8572-4d23-af72-d2174a80069f None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:11.259285 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f046f1-8572-4d23-af72-d2174a80069f None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:11.259592 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f046f1-8572-4d23-af72-d2174a80069f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:11.323401 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:11.330011 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad500a8f-0458-4c53-b4db-c7c84103d81e None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:11.336426 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f046f1-8572-4d23-af72-d2174a80069f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:11.337971 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f046f1-8572-4d23-af72-d2174a80069f None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:11.423737 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:11.424115 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd30c10-7006-44b0-a022-a53a9f85dc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:11.424688 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd30c10-7006-44b0-a022-a53a9f85dc7e None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:11.424931 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd30c10-7006-44b0-a022-a53a9f85dc7e None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:11.439445 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd30c10-7006-44b0-a022-a53a9f85dc7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:11.439445 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd30c10-7006-44b0-a022-a53a9f85dc7e None admin] RBAC: Adding query filter params (role.id=7933c45dcdd54ef0a98018ba8a8bddaa, scope.project.id=c974a924a68947af9e8abc10c2614ae7, user.id=b17271447780411384122cb456a89cb0) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:11.439445 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd30c10-7006-44b0-a022-a53a9f85dc7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:11.439445 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd30c10-7006-44b0-a022-a53a9f85dc7e None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:11.441517 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 97/197] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:30:11 2022] GET /identity/v3/role_assignments?user.id=b17271447780411384122cb456a89cb0&scope.project.id=c974a924a68947af9e8abc10c2614ae7&role.id=7933c45dcdd54ef0a98018ba8a8bddaa => generated 259 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:11.448991 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 101/198] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:30:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 12:30:11.461290 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd30c10-7006-44b0-a022-a53a9f85dc7e None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:11.622933 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd30c10-7006-44b0-a022-a53a9f85dc7e None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:11.672340 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6936ff0c-c648-41b5-940f-47e9b88ee5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:11.672997 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6936ff0c-c648-41b5-940f-47e9b88ee5af None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:11.673277 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6936ff0c-c648-41b5-940f-47e9b88ee5af None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:11.673552 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6936ff0c-c648-41b5-940f-47e9b88ee5af None admin] PATH_INFO: `/v3/projects/c974a924a68947af9e8abc10c2614ae7` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:11.675708 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6936ff0c-c648-41b5-940f-47e9b88ee5af None admin] RBAC: Authorizing `identity:get_project(project_id=c974a924a68947af9e8abc10c2614ae7)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:11.677162 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6936ff0c-c648-41b5-940f-47e9b88ee5af None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:11.679700 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 98/199] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:30:11 2022] GET /identity/v3/projects/c974a924a68947af9e8abc10c2614ae7 => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:11.685495 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f046f1-8572-4d23-af72-d2174a80069f None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:11.809171 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f046f1-8572-4d23-af72-d2174a80069f None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:11.851410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9754b45-acf4-44d6-9eef-6f6516b7cc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:11.852044 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9754b45-acf4-44d6-9eef-6f6516b7cc59 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:11.852324 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9754b45-acf4-44d6-9eef-6f6516b7cc59 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:11.852596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9754b45-acf4-44d6-9eef-6f6516b7cc59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:11.853404 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9754b45-acf4-44d6-9eef-6f6516b7cc59 None admin] RBAC: Adding query filter params (role.id=73b935a1ebc448899bbfc2381168d1cd, scope.project.id=c974a924a68947af9e8abc10c2614ae7, user.id=b17271447780411384122cb456a89cb0) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:11.853717 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9754b45-acf4-44d6-9eef-6f6516b7cc59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:11.855237 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9754b45-acf4-44d6-9eef-6f6516b7cc59 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:11.862200 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 102/200] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:30:11 2022] GET /identity/v3/role_assignments?user.id=b17271447780411384122cb456a89cb0&scope.project.id=c974a924a68947af9e8abc10c2614ae7&role.id=73b935a1ebc448899bbfc2381168d1cd => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:12.145256 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef7144-7e7e-4361-ac26-c0b6e43e410a None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:12.145595 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef7144-7e7e-4361-ac26-c0b6e43e410a None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:12.145778 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef7144-7e7e-4361-ac26-c0b6e43e410a None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:12.147139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 99/201] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:12.153701 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d87575e-f3a1-4fa2-9280-13daeb1df594 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:12.153993 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d87575e-f3a1-4fa2-9280-13daeb1df594 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:12.154272 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d87575e-f3a1-4fa2-9280-13daeb1df594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:12.176724 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-5d87575e-f3a1-4fa2-9280-13daeb1df594 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:12.178522 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-5d87575e-f3a1-4fa2-9280-13daeb1df594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:12.223015 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:12.269924 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 103/202] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 12:30:12.587827 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef7144-7e7e-4361-ac26-c0b6e43e410a None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:12.634066 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:12.710104 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef7144-7e7e-4361-ac26-c0b6e43e410a None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:12.745470 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d87d30-da85-461d-98e7-dec701fae589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:12.746110 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d87d30-da85-461d-98e7-dec701fae589 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:12.746391 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d87d30-da85-461d-98e7-dec701fae589 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:12.746664 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d87d30-da85-461d-98e7-dec701fae589 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:12.758018 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d87d30-da85-461d-98e7-dec701fae589 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:12.763075 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-70d87d30-da85-461d-98e7-dec701fae589 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 12:30:12.763075 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:12.763075 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:12.763075 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:12.763075 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:12.763075 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:12.763075 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:12.763075 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:12.763075 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:12.763075 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:12.763075 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:12.763075 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:12.763075 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:12.763075 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:12.763075 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:12.763075 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:12.765206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:12.765206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:12.765206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:12.765206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:12.765206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:12.765206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:12.765206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:12.765206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:12.765206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:12.765206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:12.765206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:12.765206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:12.765206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:12.765206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:12.765206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:12.765206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:12.767136 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:12.767136 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:12.767136 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:12.767136 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:12.767136 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:12.767136 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:12.767136 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:12.767136 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:12.767136 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:12.767136 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:12.767136 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 12:30:12.767136 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:30:12.767136 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 100/203] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:30:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:30:12.775419 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d87575e-f3a1-4fa2-9280-13daeb1df594 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:12.905423 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d87575e-f3a1-4fa2-9280-13daeb1df594 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:12.987085 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b727dda-4580-47c7-8d35-6f5e3d960a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:12.988093 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b727dda-4580-47c7-8d35-6f5e3d960a2e None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:12.988572 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b727dda-4580-47c7-8d35-6f5e3d960a2e None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:12.989061 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b727dda-4580-47c7-8d35-6f5e3d960a2e None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:12.989985 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b727dda-4580-47c7-8d35-6f5e3d960a2e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:12.990487 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b727dda-4580-47c7-8d35-6f5e3d960a2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:12.992073 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b727dda-4580-47c7-8d35-6f5e3d960a2e None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:13.007185 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 104/204] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 12:30:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:13.012927 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d87d30-da85-461d-98e7-dec701fae589 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:13.170146 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:13.182087 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d87d30-da85-461d-98e7-dec701fae589 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:13.225426 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29e0f5b-7df1-470e-8bf3-536b29aa9368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:13.226072 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e0f5b-7df1-470e-8bf3-536b29aa9368 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:13.226350 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e0f5b-7df1-470e-8bf3-536b29aa9368 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:13.226623 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e0f5b-7df1-470e-8bf3-536b29aa9368 None admin] PATH_INFO: `/v3/users/5c053331f4ae4779912272eadee0969a` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:13.232357 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29e0f5b-7df1-470e-8bf3-536b29aa9368 None admin] RBAC: Authorizing `identity:get_user(user_id=5c053331f4ae4779912272eadee0969a)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:13.236329 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29e0f5b-7df1-470e-8bf3-536b29aa9368 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:13.236329 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 101/205] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:30:13 2022] GET /identity/v3/users/5c053331f4ae4779912272eadee0969a => generated 289 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:13.242102 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b727dda-4580-47c7-8d35-6f5e3d960a2e None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:13.313737 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:13.368959 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619c5fc-7073-4809-abcb-5694be29f566 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:13.369230 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619c5fc-7073-4809-abcb-5694be29f566 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:13.369751 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619c5fc-7073-4809-abcb-5694be29f566 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:13.371334 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 102/206] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 12:30:13.386698 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e580d-a5cc-4284-91d0-d0ee2d2b1743 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:13.387435 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e580d-a5cc-4284-91d0-d0ee2d2b1743 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:13.387929 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e580d-a5cc-4284-91d0-d0ee2d2b1743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:13.397519 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b727dda-4580-47c7-8d35-6f5e3d960a2e None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:13.413061 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-2c3e580d-a5cc-4284-91d0-d0ee2d2b1743 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:13.414473 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-2c3e580d-a5cc-4284-91d0-d0ee2d2b1743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:13.434777 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8471e11f-0d78-4763-a443-9d001a3ab40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:13.435334 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471e11f-0d78-4763-a443-9d001a3ab40a None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:13.435628 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471e11f-0d78-4763-a443-9d001a3ab40a None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:13.438425 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471e11f-0d78-4763-a443-9d001a3ab40a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:13.492926 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 103/207] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 12:30:13.503784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3784ce-5e10-4784-a7fe-7600ebf3e2f7 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:13.504336 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3784ce-5e10-4784-a7fe-7600ebf3e2f7 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:13.504744 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3784ce-5e10-4784-a7fe-7600ebf3e2f7 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:13.506305 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 104/208] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:13 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:13.525557 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70083d-b7a8-478a-8925-80e29f374030 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:13.526179 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70083d-b7a8-478a-8925-80e29f374030 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:13.526671 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70083d-b7a8-478a-8925-80e29f374030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:13.543470 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8471e11f-0d78-4763-a443-9d001a3ab40a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:13.543470 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8471e11f-0d78-4763-a443-9d001a3ab40a None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:13.578611 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-5a70083d-b7a8-478a-8925-80e29f374030 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:13.580413 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-5a70083d-b7a8-478a-8925-80e29f374030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:13.593162 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:13.702939 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 105/209] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:30:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 12:30:13.720590 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8471e11f-0d78-4763-a443-9d001a3ab40a None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:13.724106 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 105/210] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 12:30:13.881279 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a70083d-b7a8-478a-8925-80e29f374030 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:13.930836 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:13.986646 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8471e11f-0d78-4763-a443-9d001a3ab40a None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:14.053135 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db043ec-fe8c-45b4-9d58-47e522fc76d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:14.054603 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db043ec-fe8c-45b4-9d58-47e522fc76d7 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:14.055108 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db043ec-fe8c-45b4-9d58-47e522fc76d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:14.056625 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db043ec-fe8c-45b4-9d58-47e522fc76d7 None admin] PATH_INFO: `/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:14.058574 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db043ec-fe8c-45b4-9d58-47e522fc76d7 None admin] RBAC: Authorizing `identity:get_project(project_id=6ee4b811a8dc4bc39e6ed0d4275b1782)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:14.060735 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db043ec-fe8c-45b4-9d58-47e522fc76d7 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:14.064108 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 106/211] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:30:13 2022] GET /identity/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782 => generated 298 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:30:14.072064 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db043ec-fe8c-45b4-9d58-47e522fc76d7 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:14.081418 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:14.094767 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a70083d-b7a8-478a-8925-80e29f374030 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:14.167361 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aecbe7-deeb-4b4a-9fc1-1ff3ca5cdba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:14.167361 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aecbe7-deeb-4b4a-9fc1-1ff3ca5cdba4 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:14.167361 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aecbe7-deeb-4b4a-9fc1-1ff3ca5cdba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:14.167705 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aecbe7-deeb-4b4a-9fc1-1ff3ca5cdba4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:14.171337 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aecbe7-deeb-4b4a-9fc1-1ff3ca5cdba4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:14.171337 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aecbe7-deeb-4b4a-9fc1-1ff3ca5cdba4 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:14.172676 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 106/212] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:30:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:30:14.173610 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aecbe7-deeb-4b4a-9fc1-1ff3ca5cdba4 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:14.237416 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db043ec-fe8c-45b4-9d58-47e522fc76d7 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:14.284932 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d6ae48-e223-4487-b3f2-4272d12b841b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:14.285557 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d6ae48-e223-4487-b3f2-4272d12b841b None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:14.285847 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d6ae48-e223-4487-b3f2-4272d12b841b None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:14.286119 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d6ae48-e223-4487-b3f2-4272d12b841b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:14.286903 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d6ae48-e223-4487-b3f2-4272d12b841b None admin] RBAC: Adding query filter params (role.id=5c094fe41dc347cbbb61cb9342eee721, scope.project.id=6ee4b811a8dc4bc39e6ed0d4275b1782, user.id=5c053331f4ae4779912272eadee0969a) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:14.287481 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d6ae48-e223-4487-b3f2-4272d12b841b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:14.288972 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d6ae48-e223-4487-b3f2-4272d12b841b None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:14.295771 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 107/213] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:30:14 2022] GET /identity/v3/role_assignments?user.id=5c053331f4ae4779912272eadee0969a&scope.project.id=6ee4b811a8dc4bc39e6ed0d4275b1782&role.id=5c094fe41dc347cbbb61cb9342eee721 => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:30:14.296964 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d6ae48-e223-4487-b3f2-4272d12b841b None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:14.374718 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:14.395854 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aecbe7-deeb-4b4a-9fc1-1ff3ca5cdba4 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:14.418788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:14.440587 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368efe00-cbfb-4188-9708-0a207278b8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:14.440865 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368efe00-cbfb-4188-9708-0a207278b8f0 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:14.441150 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368efe00-cbfb-4188-9708-0a207278b8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:14.441422 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368efe00-cbfb-4188-9708-0a207278b8f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:14.454325 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368efe00-cbfb-4188-9708-0a207278b8f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:14.460320 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-368efe00-cbfb-4188-9708-0a207278b8f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 12:30:14.460320 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:14.460320 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:14.460320 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:14.460320 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:14.460320 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:14.460320 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:14.460320 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:14.460320 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:14.460320 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:14.460320 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:14.460320 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:14.460320 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:14.460320 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:14.460320 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:14.460320 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:14.461450 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:14.461450 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:14.461450 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:14.461450 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:14.461450 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:14.461450 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:14.461450 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:14.461450 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:14.461450 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:14.461450 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:14.461450 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:14.461450 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:14.461450 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:14.461450 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:14.461450 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:14.461450 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:14.462279 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:14.462279 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:14.462279 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:14.462279 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:14.462279 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:14.462279 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:14.462279 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:14.462279 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:14.462279 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:14.462279 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:14.462279 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 12:30:14.462279 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:30:14.466799 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 107/214] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:30:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:30:14.467189 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d6ae48-e223-4487-b3f2-4272d12b841b None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:14.468327 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e36d7-dd6d-49aa-a215-56514c63461f None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:14.468618 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e36d7-dd6d-49aa-a215-56514c63461f None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:14.468897 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e36d7-dd6d-49aa-a215-56514c63461f None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:14.474465 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 108/215] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:14.476043 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e36d7-dd6d-49aa-a215-56514c63461f None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:14.508644 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb7b940-8291-4500-b913-85dfe33055bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:14.509529 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7b940-8291-4500-b913-85dfe33055bd None admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:14.509986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7b940-8291-4500-b913-85dfe33055bd None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:14.510497 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7b940-8291-4500-b913-85dfe33055bd None admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:14.511442 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb7b940-8291-4500-b913-85dfe33055bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:14.513183 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb7b940-8291-4500-b913-85dfe33055bd None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:14.594182 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e36d7-dd6d-49aa-a215-56514c63461f None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:14.624306 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 108/216] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Oct 3 12:30:14 2022] POST /identity/v3/users => generated 297 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:30:14.630077 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf5d89-1544-47f4-a242-9898dbf12402 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:14.630770 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf5d89-1544-47f4-a242-9898dbf12402 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:14.631745 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf5d89-1544-47f4-a242-9898dbf12402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:14.635412 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad19428-4030-499e-96ce-2cef20f1c1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:14.636414 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad19428-4030-499e-96ce-2cef20f1c1e5 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:14.636900 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad19428-4030-499e-96ce-2cef20f1c1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:14.637398 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad19428-4030-499e-96ce-2cef20f1c1e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:14.638372 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad19428-4030-499e-96ce-2cef20f1c1e5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:14.639000 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad19428-4030-499e-96ce-2cef20f1c1e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:14.640877 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad19428-4030-499e-96ce-2cef20f1c1e5 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:14.651771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 109/217] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 12:30:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:14.658021 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-3ccf5d89-1544-47f4-a242-9898dbf12402 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:14.658021 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-3ccf5d89-1544-47f4-a242-9898dbf12402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:14.660711 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad19428-4030-499e-96ce-2cef20f1c1e5 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:14.679570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:14.754615 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 109/218] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:30:14.799578 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad19428-4030-499e-96ce-2cef20f1c1e5 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:14.837427 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755ff84-62e9-432f-b915-c5d8e97f9bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:14.838072 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755ff84-62e9-432f-b915-c5d8e97f9bc4 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:14.838353 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755ff84-62e9-432f-b915-c5d8e97f9bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:14.838630 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755ff84-62e9-432f-b915-c5d8e97f9bc4 None admin] PATH_INFO: `/v3/users/362c446f51cd4a3e92e8721fa400383b` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:14.844287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8b378-7572-4424-9a63-a5cd240406ff None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:14.844696 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8b378-7572-4424-9a63-a5cd240406ff None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:14.845074 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8b378-7572-4424-9a63-a5cd240406ff None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:14.846547 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 110/219] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:14.852251 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde301f-191f-4d15-a5b7-c685b52a40ac None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:14.852538 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde301f-191f-4d15-a5b7-c685b52a40ac None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:14.852937 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde301f-191f-4d15-a5b7-c685b52a40ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:14.854505 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a755ff84-62e9-432f-b915-c5d8e97f9bc4 None admin] RBAC: Authorizing `identity:get_user(user_id=362c446f51cd4a3e92e8721fa400383b)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:14.856008 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a755ff84-62e9-432f-b915-c5d8e97f9bc4 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:14.857798 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 110/220] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:30:14 2022] GET /identity/v3/users/362c446f51cd4a3e92e8721fa400383b => generated 297 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:14.863107 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755ff84-62e9-432f-b915-c5d8e97f9bc4 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:14.882155 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-cfde301f-191f-4d15-a5b7-c685b52a40ac None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:14.883603 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-cfde301f-191f-4d15-a5b7-c685b52a40ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:14.950238 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:14.958147 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:14.985557 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 111/221] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 12:30:14.993772 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755ff84-62e9-432f-b915-c5d8e97f9bc4 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:15.033018 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c5444-cf45-409f-a815-567b72e67bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:15.034006 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c5444-cf45-409f-a815-567b72e67bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:15.034006 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c5444-cf45-409f-a815-567b72e67bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:15.034250 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c5444-cf45-409f-a815-567b72e67bc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:15.056140 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfde301f-191f-4d15-a5b7-c685b52a40ac None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:15.117872 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c5444-cf45-409f-a815-567b72e67bc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:15.119752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c5444-cf45-409f-a815-567b72e67bc7 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:15.218858 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfde301f-191f-4d15-a5b7-c685b52a40ac None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:15.228808 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:15.277624 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 111/222] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:30:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 12:30:15.283862 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c5444-cf45-409f-a815-567b72e67bc7 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:15.303120 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d47a15-453f-47c6-bd33-adc500d553af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:15.304591 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d47a15-453f-47c6-bd33-adc500d553af None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:15.304866 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d47a15-453f-47c6-bd33-adc500d553af None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:15.305253 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d47a15-453f-47c6-bd33-adc500d553af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:15.322736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d47a15-453f-47c6-bd33-adc500d553af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:15.328328 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-b8d47a15-453f-47c6-bd33-adc500d553af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:30:15.328328 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:15.328328 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:15.328328 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:15.328328 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:15.328328 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:15.328328 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:15.328328 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:15.328328 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:15.328328 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:15.328328 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:15.328328 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:15.328328 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:15.328328 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:15.328328 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:15.328328 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:15.329215 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:15.329215 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:15.329215 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:15.329215 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:15.329215 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:15.329215 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:15.329215 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:15.329215 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:15.329215 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:15.329215 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:15.329215 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:15.329215 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:15.329215 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:15.329215 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:15.329215 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:15.329215 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:15.329985 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:15.329985 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:15.329985 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:15.329985 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:15.329985 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:15.329985 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:15.329985 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:15.329985 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:15.329985 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:15.329985 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:15.329985 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:30:15.329985 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:30:15.329985 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 112/223] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 12:30:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:30:15.337020 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d47a15-453f-47c6-bd33-adc500d553af None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:15.384215 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:15.442987 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c5444-cf45-409f-a815-567b72e67bc7 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:15.452085 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d47a15-453f-47c6-bd33-adc500d553af None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:15.479825 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8490c3c0-8320-414a-a03e-c8658a1c48df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:15.480612 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8490c3c0-8320-414a-a03e-c8658a1c48df None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:15.480956 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8490c3c0-8320-414a-a03e-c8658a1c48df None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:15.481346 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8490c3c0-8320-414a-a03e-c8658a1c48df None admin] PATH_INFO: `/v3/projects/c974a924a68947af9e8abc10c2614ae7` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:15.482576 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8490c3c0-8320-414a-a03e-c8658a1c48df None admin] RBAC: Authorizing `identity:get_project(project_id=c974a924a68947af9e8abc10c2614ae7)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:15.484101 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8490c3c0-8320-414a-a03e-c8658a1c48df None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:15.485897 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 112/224] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:30:15 2022] GET /identity/v3/projects/c974a924a68947af9e8abc10c2614ae7 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:30:15.486472 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8490c3c0-8320-414a-a03e-c8658a1c48df None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:15.495223 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fcb685-0c76-4336-8d12-a75c35d79dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:15.495912 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcb685-0c76-4336-8d12-a75c35d79dd3 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:15.496189 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcb685-0c76-4336-8d12-a75c35d79dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:15.496458 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcb685-0c76-4336-8d12-a75c35d79dd3 None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:15.497124 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fcb685-0c76-4336-8d12-a75c35d79dd3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:15.497424 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fcb685-0c76-4336-8d12-a75c35d79dd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:15.498876 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fcb685-0c76-4336-8d12-a75c35d79dd3 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:15.510061 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 113/225] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:30:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:30:15.510715 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fcb685-0c76-4336-8d12-a75c35d79dd3 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:15.536172 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:15.622708 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fcb685-0c76-4336-8d12-a75c35d79dd3 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:15.624977 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:15.629651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8490c3c0-8320-414a-a03e-c8658a1c48df None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:15.658539 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe713e-21ae-441b-b491-ca6190c1f99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:15.659478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe713e-21ae-441b-b491-ca6190c1f99f None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:15.659598 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe713e-21ae-441b-b491-ca6190c1f99f None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:15.659941 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe713e-21ae-441b-b491-ca6190c1f99f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:15.660730 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fe713e-21ae-441b-b491-ca6190c1f99f None admin] RBAC: Adding query filter params (role.id=47d5e2ea167241d5a27c8bbf4593eead, scope.project.id=c974a924a68947af9e8abc10c2614ae7, user.id=362c446f51cd4a3e92e8721fa400383b) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:15.661313 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fe713e-21ae-441b-b491-ca6190c1f99f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:15.663952 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fe713e-21ae-441b-b491-ca6190c1f99f None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:15.670330 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba1583f-ca02-4f4a-b5f9-084fd96362d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:15.670527 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 114/226] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:30:15 2022] GET /identity/v3/role_assignments?user.id=362c446f51cd4a3e92e8721fa400383b&scope.project.id=c974a924a68947af9e8abc10c2614ae7&role.id=47d5e2ea167241d5a27c8bbf4593eead => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:15.671335 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1583f-ca02-4f4a-b5f9-084fd96362d2 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:15.671544 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe713e-21ae-441b-b491-ca6190c1f99f None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:15.671910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1583f-ca02-4f4a-b5f9-084fd96362d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:15.672364 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1583f-ca02-4f4a-b5f9-084fd96362d2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:15.685992 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba1583f-ca02-4f4a-b5f9-084fd96362d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:15.691838 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-eba1583f-ca02-4f4a-b5f9-084fd96362d2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 12:30:15.691838 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:15.691838 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:15.691838 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:15.691838 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:15.691838 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:15.691838 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:15.691838 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:15.691838 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:15.691838 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:15.691838 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:15.691838 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:15.691838 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:15.691838 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:15.691838 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:15.691838 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:15.692757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:15.692757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:15.692757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:15.692757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:15.692757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:15.692757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:15.692757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:15.692757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:15.692757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:15.692757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:15.692757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:15.692757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:15.692757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:15.692757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:15.692757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:15.692757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:15.694139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:15.694139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:15.694139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:15.694139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:15.694139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:15.694139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:15.694139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:15.694139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:15.694139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:15.694139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:15.694139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 12:30:15.694139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:30:15.696476 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 113/227] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 12:30:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:30:15.708699 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba1583f-ca02-4f4a-b5f9-084fd96362d2 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:15.783612 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe713e-21ae-441b-b491-ca6190c1f99f None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:15.820902 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e01509-5d18-4d9d-8eda-86335ccb090e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:15.821734 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e01509-5d18-4d9d-8eda-86335ccb090e None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:15.822111 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e01509-5d18-4d9d-8eda-86335ccb090e None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:15.822484 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e01509-5d18-4d9d-8eda-86335ccb090e None admin] PATH_INFO: `/v3/users/b17271447780411384122cb456a89cb0` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:15.823853 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e01509-5d18-4d9d-8eda-86335ccb090e None admin] RBAC: Authorizing `identity:get_user(user_id=b17271447780411384122cb456a89cb0)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:15.825396 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e01509-5d18-4d9d-8eda-86335ccb090e None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:15.827613 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 115/228] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:30:15 2022] GET /identity/v3/users/b17271447780411384122cb456a89cb0 => generated 283 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:15.833255 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e01509-5d18-4d9d-8eda-86335ccb090e None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:15.875665 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:15.902827 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:15.911331 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba1583f-ca02-4f4a-b5f9-084fd96362d2 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:15.950731 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab39cc-22d9-4f6a-b29e-bb0355ad345b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:15.951217 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e01509-5d18-4d9d-8eda-86335ccb090e None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:15.951840 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab39cc-22d9-4f6a-b29e-bb0355ad345b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:15.952270 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab39cc-22d9-4f6a-b29e-bb0355ad345b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:15.952686 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab39cc-22d9-4f6a-b29e-bb0355ad345b None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:15.953656 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab39cc-22d9-4f6a-b29e-bb0355ad345b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:15.954232 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab39cc-22d9-4f6a-b29e-bb0355ad345b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:15.956094 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab39cc-22d9-4f6a-b29e-bb0355ad345b None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:15.966956 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 114/229] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:30:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:15.972891 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab39cc-22d9-4f6a-b29e-bb0355ad345b None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:15.990196 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d5475-bafa-40fb-9e30-e1051d4b8a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:15.991004 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d5475-bafa-40fb-9e30-e1051d4b8a58 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:15.991286 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d5475-bafa-40fb-9e30-e1051d4b8a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:15.991582 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d5475-bafa-40fb-9e30-e1051d4b8a58 None admin] PATH_INFO: `/v3/projects/c974a924a68947af9e8abc10c2614ae7` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:15.993128 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318d5475-bafa-40fb-9e30-e1051d4b8a58 None admin] RBAC: Authorizing `identity:get_project(project_id=c974a924a68947af9e8abc10c2614ae7)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:15.994476 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318d5475-bafa-40fb-9e30-e1051d4b8a58 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:15.996245 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 116/230] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:30:15 2022] GET /identity/v3/projects/c974a924a68947af9e8abc10c2614ae7 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:16.001946 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d5475-bafa-40fb-9e30-e1051d4b8a58 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:16.087932 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab39cc-22d9-4f6a-b29e-bb0355ad345b None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:16.107066 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d5475-bafa-40fb-9e30-e1051d4b8a58 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:16.109380 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:16.148646 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d1a10-69ae-459b-bf97-794333fdd32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:16.149259 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d1a10-69ae-459b-bf97-794333fdd32a None admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:16.149556 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d1a10-69ae-459b-bf97-794333fdd32a None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:16.149818 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d1a10-69ae-459b-bf97-794333fdd32a None admin] PATH_INFO: `/v3/projects/c974a924a68947af9e8abc10c2614ae7/users/b17271447780411384122cb456a89cb0/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:16.154232 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d1a10-69ae-459b-bf97-794333fdd32a None admin] RBAC: Authorizing `identity:create_grant(project_id=c974a924a68947af9e8abc10c2614ae7, user_id=b17271447780411384122cb456a89cb0, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:16.154232 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d1a10-69ae-459b-bf97-794333fdd32a None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:16.156144 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a94340f-6684-4c55-a0de-5e31261809fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:16.157018 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a94340f-6684-4c55-a0de-5e31261809fc None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:16.157498 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a94340f-6684-4c55-a0de-5e31261809fc None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:16.158054 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a94340f-6684-4c55-a0de-5e31261809fc None admin] PATH_INFO: `/v3/users/b17271447780411384122cb456a89cb0` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:16.164022 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a94340f-6684-4c55-a0de-5e31261809fc None admin] RBAC: Authorizing `identity:get_user(user_id=b17271447780411384122cb456a89cb0)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:16.165471 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a94340f-6684-4c55-a0de-5e31261809fc None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:16.171367 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 115/231] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:30:15 2022] GET /identity/v3/users/b17271447780411384122cb456a89cb0 => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:16.172599 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a94340f-6684-4c55-a0de-5e31261809fc None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:16.208444 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:16.230061 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 117/232] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 12:30:16 2022] PUT /identity/v3/projects/c974a924a68947af9e8abc10c2614ae7/users/b17271447780411384122cb456a89cb0/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:30:16.293677 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a94340f-6684-4c55-a0de-5e31261809fc None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:16.332387 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e3f3f1-0c39-47ff-8831-7ec566ca5f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:16.333375 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3f3f1-0c39-47ff-8831-7ec566ca5f27 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:16.334400 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3f3f1-0c39-47ff-8831-7ec566ca5f27 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:16.334821 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3f3f1-0c39-47ff-8831-7ec566ca5f27 None admin] PATH_INFO: `/v3/projects/c974a924a68947af9e8abc10c2614ae7` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:16.336249 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e3f3f1-0c39-47ff-8831-7ec566ca5f27 None admin] RBAC: Authorizing `identity:get_project(project_id=c974a924a68947af9e8abc10c2614ae7)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:16.337970 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e3f3f1-0c39-47ff-8831-7ec566ca5f27 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:16.340496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 116/233] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:30:16 2022] GET /identity/v3/projects/c974a924a68947af9e8abc10c2614ae7 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:16.348737 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d1a10-69ae-459b-bf97-794333fdd32a None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:16.514999 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d1a10-69ae-459b-bf97-794333fdd32a None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:16.556296 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3517998-0595-4baf-824e-d69c6811a974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:16.557206 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3517998-0595-4baf-824e-d69c6811a974 None admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:16.557661 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3517998-0595-4baf-824e-d69c6811a974 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:16.558079 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3517998-0595-4baf-824e-d69c6811a974 None admin] PATH_INFO: `/v3/projects/c974a924a68947af9e8abc10c2614ae7/users/b17271447780411384122cb456a89cb0/roles/73b935a1ebc448899bbfc2381168d1cd` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:16.560441 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3517998-0595-4baf-824e-d69c6811a974 None admin] RBAC: Authorizing `identity:create_grant(project_id=c974a924a68947af9e8abc10c2614ae7, user_id=b17271447780411384122cb456a89cb0, role_id=73b935a1ebc448899bbfc2381168d1cd)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:16.562238 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3517998-0595-4baf-824e-d69c6811a974 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:16.669292 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 118/234] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 12:30:16 2022] PUT /identity/v3/projects/c974a924a68947af9e8abc10c2614ae7/users/b17271447780411384122cb456a89cb0/roles/73b935a1ebc448899bbfc2381168d1cd => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:30:17.230420 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98422aab-3115-4d94-830e-bd1c99bca76c None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:17.231269 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98422aab-3115-4d94-830e-bd1c99bca76c None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:17.231714 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98422aab-3115-4d94-830e-bd1c99bca76c None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:17.233215 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 117/235] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:17.238745 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1342f3-ed8c-4af1-8e46-56fa8ee628eb None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:17.239560 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1342f3-ed8c-4af1-8e46-56fa8ee628eb None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:17.240658 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1342f3-ed8c-4af1-8e46-56fa8ee628eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:17.268588 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-2c1342f3-ed8c-4af1-8e46-56fa8ee628eb None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:17.269787 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-2c1342f3-ed8c-4af1-8e46-56fa8ee628eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:17.296421 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:17.355369 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 119/236] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:30:17.539980 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05412f52-26c0-4826-8235-f7b1fb103585 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:17.540305 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05412f52-26c0-4826-8235-f7b1fb103585 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:17.540534 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05412f52-26c0-4826-8235-f7b1fb103585 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:17.541888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 118/237] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:17.547248 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3c071-8c32-4485-816c-db3c64480e8e None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:17.547581 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3c071-8c32-4485-816c-db3c64480e8e None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:17.547861 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3c071-8c32-4485-816c-db3c64480e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:17.567754 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-fed3c071-8c32-4485-816c-db3c64480e8e None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:17.569604 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-fed3c071-8c32-4485-816c-db3c64480e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:17.659894 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 120/238] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:30:17.672620 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05412f52-26c0-4826-8235-f7b1fb103585 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:17.775099 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:17.784738 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05412f52-26c0-4826-8235-f7b1fb103585 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:17.820077 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b13b0c-0fcc-44f1-a4c1-82be8cddc272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:17.820641 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b13b0c-0fcc-44f1-a4c1-82be8cddc272 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:17.820933 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b13b0c-0fcc-44f1-a4c1-82be8cddc272 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:17.821423 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b13b0c-0fcc-44f1-a4c1-82be8cddc272 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:17.822533 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b13b0c-0fcc-44f1-a4c1-82be8cddc272 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:17.824037 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b13b0c-0fcc-44f1-a4c1-82be8cddc272 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:17.830346 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 119/239] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:30:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:17.831543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed3c071-8c32-4485-816c-db3c64480e8e None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:17.972860 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:17.992390 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b13b0c-0fcc-44f1-a4c1-82be8cddc272 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:17.995077 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed3c071-8c32-4485-816c-db3c64480e8e None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:18.034347 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc1b51a-c225-4e32-ae8b-cb5c6866ed5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:18.034956 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1b51a-c225-4e32-ae8b-cb5c6866ed5c None admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:18.035231 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1b51a-c225-4e32-ae8b-cb5c6866ed5c None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:18.035526 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1b51a-c225-4e32-ae8b-cb5c6866ed5c None admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:18.036167 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc1b51a-c225-4e32-ae8b-cb5c6866ed5c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:18.040747 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc1b51a-c225-4e32-ae8b-cb5c6866ed5c None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:18.125740 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 121/240] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Oct 3 12:30:17 2022] POST /identity/v3/users => generated 293 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:30:18.147084 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b13b0c-0fcc-44f1-a4c1-82be8cddc272 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:18.192464 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf12aaa-bda9-48a0-84ba-495a4dc260df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:18.193221 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf12aaa-bda9-48a0-84ba-495a4dc260df None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:18.193505 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf12aaa-bda9-48a0-84ba-495a4dc260df None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:18.193777 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf12aaa-bda9-48a0-84ba-495a4dc260df None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:18.209766 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf12aaa-bda9-48a0-84ba-495a4dc260df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:18.214649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-acf12aaa-bda9-48a0-84ba-495a4dc260df None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 12:30:18.214649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:18.214649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:18.214649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:18.214649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:18.214649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:18.214649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:18.214649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:18.214649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:18.214649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:18.214649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:18.214649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:18.214649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:18.214649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:18.214649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:18.214649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:18.215824 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:18.215824 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:18.215824 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:18.215824 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:18.215824 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:18.215824 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:18.215824 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:18.215824 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:18.215824 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:18.215824 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:18.215824 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:18.215824 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:18.215824 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:18.215824 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:18.215824 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:18.215824 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:18.216585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:18.216585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:18.216585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:18.216585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:18.216585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:18.216585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:18.216585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:18.216585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:18.216585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:18.216585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:18.216585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 12:30:18.216585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:30:18.216585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 120/241] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:30:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:30:18.222142 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc1b51a-c225-4e32-ae8b-cb5c6866ed5c None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:18.390575 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc1b51a-c225-4e32-ae8b-cb5c6866ed5c None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:18.433886 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bfe045-5939-4c8c-8b15-13cd3336f677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:18.434884 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bfe045-5939-4c8c-8b15-13cd3336f677 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:18.435399 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bfe045-5939-4c8c-8b15-13cd3336f677 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:18.435875 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bfe045-5939-4c8c-8b15-13cd3336f677 None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:18.436732 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bfe045-5939-4c8c-8b15-13cd3336f677 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:18.437227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bfe045-5939-4c8c-8b15-13cd3336f677 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:18.438783 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bfe045-5939-4c8c-8b15-13cd3336f677 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:18.451462 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 122/242] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 12:30:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:18.456732 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf12aaa-bda9-48a0-84ba-495a4dc260df None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:18.494323 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:18.598276 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf12aaa-bda9-48a0-84ba-495a4dc260df None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:18.638170 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e11282-0145-4433-8fa4-c613dbab1241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:18.638868 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e11282-0145-4433-8fa4-c613dbab1241 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:18.639190 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e11282-0145-4433-8fa4-c613dbab1241 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:18.639499 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e11282-0145-4433-8fa4-c613dbab1241 None admin] PATH_INFO: `/v3/users/74a000af0aa741488d5a21ee9d974ea5` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:18.653762 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e11282-0145-4433-8fa4-c613dbab1241 None admin] RBAC: Authorizing `identity:get_user(user_id=74a000af0aa741488d5a21ee9d974ea5)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:18.655160 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e11282-0145-4433-8fa4-c613dbab1241 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:18.657141 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 121/243] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:30:18 2022] GET /identity/v3/users/74a000af0aa741488d5a21ee9d974ea5 => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:18.663497 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e11282-0145-4433-8fa4-c613dbab1241 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:18.663648 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242c6cf-3674-4388-bfd9-94fcc79c42c2 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:18.663917 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242c6cf-3674-4388-bfd9-94fcc79c42c2 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:18.664186 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242c6cf-3674-4388-bfd9-94fcc79c42c2 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:18.665789 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 123/244] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:18.670706 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011adb4-f930-4d6c-b275-8ba3feb87e26 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:18.670989 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011adb4-f930-4d6c-b275-8ba3feb87e26 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:18.671263 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011adb4-f930-4d6c-b275-8ba3feb87e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:18.683729 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:18.704558 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-a011adb4-f930-4d6c-b275-8ba3feb87e26 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:18.705932 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-a011adb4-f930-4d6c-b275-8ba3feb87e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:18.773516 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:18.798113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e11282-0145-4433-8fa4-c613dbab1241 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:18.816902 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 124/245] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 12:30:18.839113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec770b-e012-423b-a42e-12822a8b6286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:18.840026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec770b-e012-423b-a42e-12822a8b6286 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:18.840707 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec770b-e012-423b-a42e-12822a8b6286 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:18.841133 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec770b-e012-423b-a42e-12822a8b6286 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:18.914880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ec770b-e012-423b-a42e-12822a8b6286 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:18.916642 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ec770b-e012-423b-a42e-12822a8b6286 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:18.996486 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:19.019193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 122/246] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:30:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 12:30:19.025768 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011adb4-f930-4d6c-b275-8ba3feb87e26 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:19.144394 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011adb4-f930-4d6c-b275-8ba3feb87e26 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:19.176118 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:19.182498 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35da0ecb-9f20-4e84-b5b8-dd8350164f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:19.183480 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35da0ecb-9f20-4e84-b5b8-dd8350164f53 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:19.183949 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35da0ecb-9f20-4e84-b5b8-dd8350164f53 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:19.191211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35da0ecb-9f20-4e84-b5b8-dd8350164f53 None admin] PATH_INFO: `/v3/projects/c974a924a68947af9e8abc10c2614ae7` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:19.191211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35da0ecb-9f20-4e84-b5b8-dd8350164f53 None admin] RBAC: Authorizing `identity:get_project(project_id=c974a924a68947af9e8abc10c2614ae7)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:19.191211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35da0ecb-9f20-4e84-b5b8-dd8350164f53 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:19.191211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 125/247] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:30:19 2022] GET /identity/v3/projects/c974a924a68947af9e8abc10c2614ae7 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:19.194642 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec770b-e012-423b-a42e-12822a8b6286 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:19.236602 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09585ca-cbf8-469d-af2f-c0daea76e956 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:19.236956 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09585ca-cbf8-469d-af2f-c0daea76e956 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:19.237308 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09585ca-cbf8-469d-af2f-c0daea76e956 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:19.238726 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 126/248] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:19.246273 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4fb88a-58a7-4d44-bece-2dbee497002e None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:19.246273 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4fb88a-58a7-4d44-bece-2dbee497002e None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:19.246273 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4fb88a-58a7-4d44-bece-2dbee497002e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:19.264917 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-7e4fb88a-58a7-4d44-bece-2dbee497002e None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:19.266480 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-7e4fb88a-58a7-4d44-bece-2dbee497002e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:19.323730 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec770b-e012-423b-a42e-12822a8b6286 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:19.344351 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 127/249] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 12:30:19.345632 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4fb88a-58a7-4d44-bece-2dbee497002e None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:19.395547 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb6886-849c-43a4-9ca4-ce0275ba9b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:19.396019 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb6886-849c-43a4-9ca4-ce0275ba9b07 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:19.396455 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb6886-849c-43a4-9ca4-ce0275ba9b07 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:19.397020 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb6886-849c-43a4-9ca4-ce0275ba9b07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:19.397790 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fb6886-849c-43a4-9ca4-ce0275ba9b07 None admin] RBAC: Adding query filter params (role.id=5c094fe41dc347cbbb61cb9342eee721, scope.project.id=c974a924a68947af9e8abc10c2614ae7, user.id=74a000af0aa741488d5a21ee9d974ea5) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:19.398164 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fb6886-849c-43a4-9ca4-ce0275ba9b07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:19.399691 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fb6886-849c-43a4-9ca4-ce0275ba9b07 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:19.426366 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 123/250] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:30:19 2022] GET /identity/v3/role_assignments?user.id=74a000af0aa741488d5a21ee9d974ea5&scope.project.id=c974a924a68947af9e8abc10c2614ae7&role.id=5c094fe41dc347cbbb61cb9342eee721 => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:30:19.494995 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4fb88a-58a7-4d44-bece-2dbee497002e None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:19.531029 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:19.537739 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d872894d-dd6d-4d7c-82a8-d3aef0cf8384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:19.538715 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872894d-dd6d-4d7c-82a8-d3aef0cf8384 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:19.539168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872894d-dd6d-4d7c-82a8-d3aef0cf8384 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:19.539637 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872894d-dd6d-4d7c-82a8-d3aef0cf8384 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:19.551393 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d872894d-dd6d-4d7c-82a8-d3aef0cf8384 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:19.558361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-d872894d-dd6d-4d7c-82a8-d3aef0cf8384 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 12:30:19.558361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:19.558361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:19.558361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:19.558361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:19.558361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:19.558361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:19.558361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:19.558361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:19.558361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:19.558361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:19.558361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:19.558361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:19.558361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:19.558361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:19.558361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:19.559311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:19.559311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:19.559311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:19.559311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:19.559311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:19.559311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:19.559311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:19.559311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:19.559311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:19.559311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:19.559311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:19.559311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:19.559311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:19.559311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:19.559311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:19.559311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:19.560123 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:19.560123 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:19.560123 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:19.560123 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:19.560123 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:19.560123 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:19.560123 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:19.560123 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:19.560123 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:19.560123 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:19.560123 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 12:30:19.560123 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:30:19.562978 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 128/251] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:30:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:30:19.576201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb6886-849c-43a4-9ca4-ce0275ba9b07 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:19.704511 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:19.768577 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d872894d-dd6d-4d7c-82a8-d3aef0cf8384 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:19.834477 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb6886-849c-43a4-9ca4-ce0275ba9b07 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:19.874609 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93416c00-51e9-412f-8ea5-0b0cf3172e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:19.875220 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93416c00-51e9-412f-8ea5-0b0cf3172e62 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:19.875528 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93416c00-51e9-412f-8ea5-0b0cf3172e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:19.875799 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93416c00-51e9-412f-8ea5-0b0cf3172e62 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:19.876461 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93416c00-51e9-412f-8ea5-0b0cf3172e62 None admin] RBAC: Adding query filter params (name=member) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:19.876760 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93416c00-51e9-412f-8ea5-0b0cf3172e62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:19.887342 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93416c00-51e9-412f-8ea5-0b0cf3172e62 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:19.887342 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 124/252] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 12:30:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:19.889185 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315ba0e-35d6-4959-903a-5798beb04a70 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:19.889471 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315ba0e-35d6-4959-903a-5798beb04a70 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:19.892547 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315ba0e-35d6-4959-903a-5798beb04a70 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:19.892547 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 125/253] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:19.897026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a315ba0e-35d6-4959-903a-5798beb04a70 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:20.073249 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d872894d-dd6d-4d7c-82a8-d3aef0cf8384 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:20.135741 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a315ba0e-35d6-4959-903a-5798beb04a70 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:20.149837 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:20.172061 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11369bec-00f3-40e9-97a6-4db4deef472d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:20.172773 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11369bec-00f3-40e9-97a6-4db4deef472d None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:20.173065 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11369bec-00f3-40e9-97a6-4db4deef472d None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:20.173341 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11369bec-00f3-40e9-97a6-4db4deef472d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:20.216526 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11369bec-00f3-40e9-97a6-4db4deef472d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:20.223724 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-11369bec-00f3-40e9-97a6-4db4deef472d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:30:20.223724 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:20.223724 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:20.223724 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:20.223724 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:20.223724 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:20.223724 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:20.223724 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:20.223724 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:20.223724 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:20.223724 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:20.223724 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:20.223724 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:20.223724 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:20.223724 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:20.223724 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:20.225293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:20.225293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:20.225293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:20.225293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:20.225293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:20.225293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:20.225293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:20.225293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:20.225293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:20.225293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:20.225293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:20.225293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:20.225293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:20.225293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:20.225293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:20.225293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:20.226223 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:20.226223 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:20.226223 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:20.226223 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:20.226223 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:20.226223 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:20.226223 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:20.226223 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:20.226223 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:20.226223 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:20.226223 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:30:20.226223 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:30:20.226223 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 129/254] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 12:30:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 455 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:30:20.226957 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f469e65-e52b-4c23-af7c-6163600d735a None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:20.227562 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f469e65-e52b-4c23-af7c-6163600d735a None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:20.227562 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f469e65-e52b-4c23-af7c-6163600d735a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:20.279348 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-8f469e65-e52b-4c23-af7c-6163600d735a None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:20.280570 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-8f469e65-e52b-4c23-af7c-6163600d735a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:20.285770 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcffa06c-f95c-49a5-b509-b8032bdca2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:20.285770 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcffa06c-f95c-49a5-b509-b8032bdca2a0 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:20.286112 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcffa06c-f95c-49a5-b509-b8032bdca2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:20.286112 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcffa06c-f95c-49a5-b509-b8032bdca2a0 None admin] PATH_INFO: `/v3/users/362c446f51cd4a3e92e8721fa400383b` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:20.288391 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcffa06c-f95c-49a5-b509-b8032bdca2a0 None admin] RBAC: Authorizing `identity:get_user(user_id=362c446f51cd4a3e92e8721fa400383b)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:20.290873 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcffa06c-f95c-49a5-b509-b8032bdca2a0 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:20.305470 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 126/255] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:30:19 2022] GET /identity/v3/users/362c446f51cd4a3e92e8721fa400383b => generated 297 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:20.306725 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcffa06c-f95c-49a5-b509-b8032bdca2a0 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:20.407519 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:20.458131 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 130/256] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:30:20.459803 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f469e65-e52b-4c23-af7c-6163600d735a None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:20.533016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcffa06c-f95c-49a5-b509-b8032bdca2a0 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:20.622106 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6040b0eb-acc7-49a1-920b-12bc4ff37370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:20.622962 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6040b0eb-acc7-49a1-920b-12bc4ff37370 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:20.623117 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6040b0eb-acc7-49a1-920b-12bc4ff37370 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:20.623307 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6040b0eb-acc7-49a1-920b-12bc4ff37370 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:20.624010 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6040b0eb-acc7-49a1-920b-12bc4ff37370 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:20.624316 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6040b0eb-acc7-49a1-920b-12bc4ff37370 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:20.625708 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6040b0eb-acc7-49a1-920b-12bc4ff37370 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:20.644225 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 127/257] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:30:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:20.647914 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6040b0eb-acc7-49a1-920b-12bc4ff37370 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:20.680157 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f469e65-e52b-4c23-af7c-6163600d735a None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:20.722058 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:20.791414 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60812065-a875-4266-95a7-934575b90cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:20.791414 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60812065-a875-4266-95a7-934575b90cae None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:20.791414 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60812065-a875-4266-95a7-934575b90cae None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:20.791414 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60812065-a875-4266-95a7-934575b90cae None admin] PATH_INFO: `/v3/projects/c974a924a68947af9e8abc10c2614ae7` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:20.791849 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60812065-a875-4266-95a7-934575b90cae None admin] RBAC: Authorizing `identity:get_project(project_id=c974a924a68947af9e8abc10c2614ae7)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:20.793215 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60812065-a875-4266-95a7-934575b90cae None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:20.798126 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 131/258] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:30:20 2022] GET /identity/v3/projects/c974a924a68947af9e8abc10c2614ae7 => generated 302 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:20.808463 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60812065-a875-4266-95a7-934575b90cae None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:20.952804 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6040b0eb-acc7-49a1-920b-12bc4ff37370 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:21.001774 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:21.065528 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d20bba-f152-41c6-ac20-c3a5ee1c8a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:21.066475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d20bba-f152-41c6-ac20-c3a5ee1c8a7d None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:21.066927 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d20bba-f152-41c6-ac20-c3a5ee1c8a7d None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:21.067421 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d20bba-f152-41c6-ac20-c3a5ee1c8a7d None admin] PATH_INFO: `/v3/users/b17271447780411384122cb456a89cb0` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:21.073536 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d20bba-f152-41c6-ac20-c3a5ee1c8a7d None admin] RBAC: Authorizing `identity:get_user(user_id=b17271447780411384122cb456a89cb0)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:21.075281 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d20bba-f152-41c6-ac20-c3a5ee1c8a7d None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:21.077421 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 128/259] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:30:20 2022] GET /identity/v3/users/b17271447780411384122cb456a89cb0 => generated 283 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:21.078456 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d20bba-f152-41c6-ac20-c3a5ee1c8a7d None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:21.094291 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60812065-a875-4266-95a7-934575b90cae None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:21.168846 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac67bb3-a99c-405f-b515-ce0535f0b777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:21.169956 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac67bb3-a99c-405f-b515-ce0535f0b777 None admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:21.170415 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac67bb3-a99c-405f-b515-ce0535f0b777 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:21.170856 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac67bb3-a99c-405f-b515-ce0535f0b777 None admin] PATH_INFO: `/v3/projects/c974a924a68947af9e8abc10c2614ae7/users/362c446f51cd4a3e92e8721fa400383b/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:21.173585 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac67bb3-a99c-405f-b515-ce0535f0b777 None admin] RBAC: Authorizing `identity:create_grant(project_id=c974a924a68947af9e8abc10c2614ae7, user_id=362c446f51cd4a3e92e8721fa400383b, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:21.176521 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac67bb3-a99c-405f-b515-ce0535f0b777 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:21.244364 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d20bba-f152-41c6-ac20-c3a5ee1c8a7d None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:21.245359 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:21.282038 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a002d299-dfeb-4fef-9fd9-b129e0756f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:21.283051 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002d299-dfeb-4fef-9fd9-b129e0756f42 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:21.283528 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002d299-dfeb-4fef-9fd9-b129e0756f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:21.284001 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002d299-dfeb-4fef-9fd9-b129e0756f42 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:21.295058 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a002d299-dfeb-4fef-9fd9-b129e0756f42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:21.303060 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-a002d299-dfeb-4fef-9fd9-b129e0756f42 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 12:30:21.303060 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:21.303060 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:21.303060 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:21.303060 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:21.303060 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:21.303060 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:21.303060 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:21.303060 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:21.303060 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:21.303060 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:21.303060 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:21.303060 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:21.303060 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:21.303060 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:21.303060 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:21.304189 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:21.304189 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:21.304189 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:21.304189 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:21.304189 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:21.304189 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:21.304189 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:21.304189 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:21.304189 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:21.304189 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:21.304189 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:21.304189 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:21.304189 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:21.304189 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:21.304189 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:21.304189 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:21.305033 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:21.305033 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:21.305033 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:21.305033 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:21.305033 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:21.305033 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:21.305033 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:21.305033 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:21.305033 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:21.305033 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:21.305033 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 12:30:21.305033 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:30:21.308105 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 129/260] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 12:30:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:30:21.311441 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a002d299-dfeb-4fef-9fd9-b129e0756f42 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:21.314037 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 132/261] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 12:30:20 2022] PUT /identity/v3/projects/c974a924a68947af9e8abc10c2614ae7/users/362c446f51cd4a3e92e8721fa400383b/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:30:21.315111 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac67bb3-a99c-405f-b515-ce0535f0b777 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:21.577898 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a002d299-dfeb-4fef-9fd9-b129e0756f42 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:21.634476 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de0838e-093f-4dbe-a6fd-37f8f8fdad2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:21.636103 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de0838e-093f-4dbe-a6fd-37f8f8fdad2e None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:21.638181 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac67bb3-a99c-405f-b515-ce0535f0b777 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:21.640127 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de0838e-093f-4dbe-a6fd-37f8f8fdad2e None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:21.640599 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de0838e-093f-4dbe-a6fd-37f8f8fdad2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:21.658163 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:21.707477 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:21.731634 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391deff8-3f38-441a-af37-256498338e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:21.732874 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391deff8-3f38-441a-af37-256498338e3a None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:21.733351 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391deff8-3f38-441a-af37-256498338e3a None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:21.733790 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391deff8-3f38-441a-af37-256498338e3a None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:21.734626 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391deff8-3f38-441a-af37-256498338e3a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:21.735097 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391deff8-3f38-441a-af37-256498338e3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:21.736766 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391deff8-3f38-441a-af37-256498338e3a None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:21.753782 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 133/262] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:30:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 439 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:21.759527 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391deff8-3f38-441a-af37-256498338e3a None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:21.774765 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de0838e-093f-4dbe-a6fd-37f8f8fdad2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:21.776254 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de0838e-093f-4dbe-a6fd-37f8f8fdad2e None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:21.951019 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391deff8-3f38-441a-af37-256498338e3a None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:21.954298 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 130/263] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:30:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 645 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 12:30:21.955251 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868dcbd6-46cf-4cea-89bb-08b84b2494f5 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:21.955567 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868dcbd6-46cf-4cea-89bb-08b84b2494f5 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:21.955840 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868dcbd6-46cf-4cea-89bb-08b84b2494f5 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:21.957353 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 131/264] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:21.959255 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fec982-556b-4650-b0c7-8be4f134dff9 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:21.959555 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fec982-556b-4650-b0c7-8be4f134dff9 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:21.959797 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fec982-556b-4650-b0c7-8be4f134dff9 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:21.961419 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 132/265] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:21.961891 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fec982-556b-4650-b0c7-8be4f134dff9 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:21.989701 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e6db9f-a054-4670-b3dc-67818a4b8ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:21.990941 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6db9f-a054-4670-b3dc-67818a4b8ca5 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:21.991465 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6db9f-a054-4670-b3dc-67818a4b8ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:21.991932 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6db9f-a054-4670-b3dc-67818a4b8ca5 None admin] PATH_INFO: `/v3/users/b17271447780411384122cb456a89cb0` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:21.993267 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e6db9f-a054-4670-b3dc-67818a4b8ca5 None admin] RBAC: Authorizing `identity:get_user(user_id=b17271447780411384122cb456a89cb0)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:21.994949 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e6db9f-a054-4670-b3dc-67818a4b8ca5 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:21.996782 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 134/266] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:30:21 2022] GET /identity/v3/users/b17271447780411384122cb456a89cb0 => generated 283 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:30:21.998088 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59975baf-b908-487f-a7f6-c5c1ae56fbff None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:21.998579 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59975baf-b908-487f-a7f6-c5c1ae56fbff None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:21.999038 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59975baf-b908-487f-a7f6-c5c1ae56fbff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:22.019113 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-59975baf-b908-487f-a7f6-c5c1ae56fbff None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:22.020661 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-59975baf-b908-487f-a7f6-c5c1ae56fbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:22.040643 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:22.052273 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:22.086676 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fec982-556b-4650-b0c7-8be4f134dff9 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:22.113419 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 135/267] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:30:22.118435 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbc866d-73f8-4d02-a6ff-19652365abc1 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:22.118435 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbc866d-73f8-4d02-a6ff-19652365abc1 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:22.118435 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbc866d-73f8-4d02-a6ff-19652365abc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:22.140097 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-cdbc866d-73f8-4d02-a6ff-19652365abc1 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:22.141551 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-cdbc866d-73f8-4d02-a6ff-19652365abc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:22.147010 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729106c5-9630-40d3-b5d5-56edab536ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:22.147927 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729106c5-9630-40d3-b5d5-56edab536ef2 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:22.148368 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729106c5-9630-40d3-b5d5-56edab536ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:22.148800 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729106c5-9630-40d3-b5d5-56edab536ef2 None admin] PATH_INFO: `/v3/projects/c974a924a68947af9e8abc10c2614ae7` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:22.150078 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729106c5-9630-40d3-b5d5-56edab536ef2 None admin] RBAC: Authorizing `identity:get_project(project_id=c974a924a68947af9e8abc10c2614ae7)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:22.160116 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729106c5-9630-40d3-b5d5-56edab536ef2 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:22.162247 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 133/268] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:30:21 2022] GET /identity/v3/projects/c974a924a68947af9e8abc10c2614ae7 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:22.163391 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729106c5-9630-40d3-b5d5-56edab536ef2 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:22.221136 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 136/269] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:30:22.221892 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbc866d-73f8-4d02-a6ff-19652365abc1 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:22.433562 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:22.436610 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729106c5-9630-40d3-b5d5-56edab536ef2 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:22.449201 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbc866d-73f8-4d02-a6ff-19652365abc1 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:22.515261 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:22.540136 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f457428f-2c64-4b67-a913-f9bd8590f97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:22.540773 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457428f-2c64-4b67-a913-f9bd8590f97f None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:22.541048 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457428f-2c64-4b67-a913-f9bd8590f97f None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:22.541316 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457428f-2c64-4b67-a913-f9bd8590f97f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:22.577189 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ec68d-4616-4feb-a475-00e7ec5350d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:22.577827 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ec68d-4616-4feb-a475-00e7ec5350d1 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:22.578106 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ec68d-4616-4feb-a475-00e7ec5350d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:22.578378 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ec68d-4616-4feb-a475-00e7ec5350d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:22.579113 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47ec68d-4616-4feb-a475-00e7ec5350d1 None admin] RBAC: Adding query filter params (role.id=7933c45dcdd54ef0a98018ba8a8bddaa, scope.project.id=c974a924a68947af9e8abc10c2614ae7, user.id=b17271447780411384122cb456a89cb0) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:22.581919 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47ec68d-4616-4feb-a475-00e7ec5350d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:22.582921 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47ec68d-4616-4feb-a475-00e7ec5350d1 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:22.592328 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 137/270] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:30:22 2022] GET /identity/v3/role_assignments?user.id=b17271447780411384122cb456a89cb0&scope.project.id=c974a924a68947af9e8abc10c2614ae7&role.id=7933c45dcdd54ef0a98018ba8a8bddaa => generated 608 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:22.593178 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ec68d-4616-4feb-a475-00e7ec5350d1 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:22.655467 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f457428f-2c64-4b67-a913-f9bd8590f97f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:22.658125 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f457428f-2c64-4b67-a913-f9bd8590f97f None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:22.748554 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ec68d-4616-4feb-a475-00e7ec5350d1 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:22.796851 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 134/271] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:30:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 634 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 12:30:22.797688 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f457428f-2c64-4b67-a913-f9bd8590f97f None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:22.799667 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db72d4-7190-4023-b8f4-bd1138e49a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:22.801915 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db72d4-7190-4023-b8f4-bd1138e49a06 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:22.802989 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db72d4-7190-4023-b8f4-bd1138e49a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:22.803815 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db72d4-7190-4023-b8f4-bd1138e49a06 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:22.819430 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db72d4-7190-4023-b8f4-bd1138e49a06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:22.827285 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-83db72d4-7190-4023-b8f4-bd1138e49a06 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 12:30:22.827285 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:22.827285 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:22.827285 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:22.827285 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:22.827285 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:22.827285 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:22.827285 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:22.827285 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:22.827285 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:22.827285 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:22.827285 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:22.827285 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:22.827285 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:22.827285 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:22.827285 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:22.828222 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:22.828222 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:22.828222 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:22.828222 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:22.828222 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:22.828222 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:22.828222 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:22.828222 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:22.828222 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:22.828222 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:22.828222 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:22.828222 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:22.828222 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:22.828222 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:22.828222 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:22.828222 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:22.829005 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:22.829005 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:22.829005 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:22.829005 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:22.829005 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:22.829005 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:22.829005 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:22.829005 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:22.829005 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:22.829005 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:22.829005 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 12:30:22.829005 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:30:22.831835 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 138/272] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:30:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:30:22.832634 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c63ce-a117-489c-9e08-459928bdfc3b None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:22.832910 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c63ce-a117-489c-9e08-459928bdfc3b None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:22.833177 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c63ce-a117-489c-9e08-459928bdfc3b None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:22.835151 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 139/273] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:22.835894 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c63ce-a117-489c-9e08-459928bdfc3b None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:22.848984 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:22.871112 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:22.938881 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f457428f-2c64-4b67-a913-f9bd8590f97f None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:22.980177 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c63ce-a117-489c-9e08-459928bdfc3b None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:22.980487 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b43b7e-5567-4eef-97ad-80d387ca1814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:22.981130 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b43b7e-5567-4eef-97ad-80d387ca1814 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:22.981409 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b43b7e-5567-4eef-97ad-80d387ca1814 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:22.981757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b43b7e-5567-4eef-97ad-80d387ca1814 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:22.982861 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b43b7e-5567-4eef-97ad-80d387ca1814 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:22.984328 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b43b7e-5567-4eef-97ad-80d387ca1814 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:22.985903 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 135/274] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:30:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:22.986764 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b43b7e-5567-4eef-97ad-80d387ca1814 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:23.016615 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a92562-5173-420a-8598-72975af0f418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:23.017001 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a92562-5173-420a-8598-72975af0f418 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:23.017281 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a92562-5173-420a-8598-72975af0f418 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:23.017555 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a92562-5173-420a-8598-72975af0f418 None admin] PATH_INFO: `/v3/projects/c974a924a68947af9e8abc10c2614ae7` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:23.018660 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a92562-5173-420a-8598-72975af0f418 None admin] RBAC: Authorizing `identity:get_project(project_id=c974a924a68947af9e8abc10c2614ae7)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:23.020124 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a92562-5173-420a-8598-72975af0f418 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:23.023539 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 140/275] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:30:22 2022] GET /identity/v3/projects/c974a924a68947af9e8abc10c2614ae7 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:23.024719 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2336a914-b1c3-4782-b69b-7fe317a996e3 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:23.024719 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2336a914-b1c3-4782-b69b-7fe317a996e3 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:23.024719 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2336a914-b1c3-4782-b69b-7fe317a996e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:23.043669 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-2336a914-b1c3-4782-b69b-7fe317a996e3 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:23.044846 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-2336a914-b1c3-4782-b69b-7fe317a996e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:23.097037 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:23.100797 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b43b7e-5567-4eef-97ad-80d387ca1814 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:23.123623 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 141/276] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:30:23.123623 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2336a914-b1c3-4782-b69b-7fe317a996e3 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:23.148467 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:23.159026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aecdd2-dd09-4ae8-9ef3-c6c97a08e5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:23.159482 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aecdd2-dd09-4ae8-9ef3-c6c97a08e5f3 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:23.160583 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aecdd2-dd09-4ae8-9ef3-c6c97a08e5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:23.160583 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aecdd2-dd09-4ae8-9ef3-c6c97a08e5f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:23.161074 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aecdd2-dd09-4ae8-9ef3-c6c97a08e5f3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:23.161380 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aecdd2-dd09-4ae8-9ef3-c6c97a08e5f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:23.163295 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aecdd2-dd09-4ae8-9ef3-c6c97a08e5f3 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:23.178265 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 136/277] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 12:30:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:23.179537 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aecdd2-dd09-4ae8-9ef3-c6c97a08e5f3 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:23.257380 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2336a914-b1c3-4782-b69b-7fe317a996e3 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:23.293818 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfcee64-fd7e-4417-8fa1-f71dee8b6ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:23.294820 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcee64-fd7e-4417-8fa1-f71dee8b6ab4 None admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:23.295349 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcee64-fd7e-4417-8fa1-f71dee8b6ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:23.295865 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcee64-fd7e-4417-8fa1-f71dee8b6ab4 None admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:23.296699 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfcee64-fd7e-4417-8fa1-f71dee8b6ab4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:23.298556 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfcee64-fd7e-4417-8fa1-f71dee8b6ab4 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:23.320451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aecdd2-dd09-4ae8-9ef3-c6c97a08e5f3 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:23.365681 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d8d990-ed42-4400-b980-848fa4ceeb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:23.365681 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8d990-ed42-4400-b980-848fa4ceeb5a None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:23.365681 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8d990-ed42-4400-b980-848fa4ceeb5a None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:23.365681 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8d990-ed42-4400-b980-848fa4ceeb5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:23.366071 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d8d990-ed42-4400-b980-848fa4ceeb5a None admin] RBAC: Adding query filter params (role.id=73b935a1ebc448899bbfc2381168d1cd, scope.project.id=c974a924a68947af9e8abc10c2614ae7, user.id=b17271447780411384122cb456a89cb0) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:23.366071 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d8d990-ed42-4400-b980-848fa4ceeb5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:23.366071 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d8d990-ed42-4400-b980-848fa4ceeb5a None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:23.375457 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 137/278] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:30:23 2022] GET /identity/v3/role_assignments?user.id=b17271447780411384122cb456a89cb0&scope.project.id=c974a924a68947af9e8abc10c2614ae7&role.id=73b935a1ebc448899bbfc2381168d1cd => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:23.375457 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d8d990-ed42-4400-b980-848fa4ceeb5a None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:23.406543 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:23.417600 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 142/279] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Oct 3 12:30:23 2022] POST /identity/v3/users => generated 293 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:30:23.418514 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfcee64-fd7e-4417-8fa1-f71dee8b6ab4 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:23.522460 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d8d990-ed42-4400-b980-848fa4ceeb5a None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:23.546392 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:23.565460 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530a63e-fe79-451b-a891-3c0df3cae9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:23.566117 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530a63e-fe79-451b-a891-3c0df3cae9f5 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:23.566433 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530a63e-fe79-451b-a891-3c0df3cae9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:23.566758 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530a63e-fe79-451b-a891-3c0df3cae9f5 None admin] PATH_INFO: `/v3/users/f9b08b3f0ce14cd5ae6f436921a45b9c` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:23.589067 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b530a63e-fe79-451b-a891-3c0df3cae9f5 None admin] RBAC: Authorizing `identity:get_user(user_id=f9b08b3f0ce14cd5ae6f436921a45b9c)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:23.591209 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b530a63e-fe79-451b-a891-3c0df3cae9f5 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:23.593276 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 138/280] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:30:23 2022] GET /identity/v3/users/f9b08b3f0ce14cd5ae6f436921a45b9c => generated 293 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:23.593844 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfcee64-fd7e-4417-8fa1-f71dee8b6ab4 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:23.601910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530a63e-fe79-451b-a891-3c0df3cae9f5 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:23.683215 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a643842-9af1-4d28-a85a-17ef33232903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:23.684536 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a643842-9af1-4d28-a85a-17ef33232903 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:23.684536 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a643842-9af1-4d28-a85a-17ef33232903 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:23.684536 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a643842-9af1-4d28-a85a-17ef33232903 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:23.709421 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a643842-9af1-4d28-a85a-17ef33232903 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:23.714525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-1a643842-9af1-4d28-a85a-17ef33232903 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 12:30:23.714525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:23.714525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:23.714525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:23.714525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:23.714525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:23.714525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:23.714525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:23.714525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:23.714525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:23.714525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:23.714525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:23.714525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:23.714525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:23.714525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:23.714525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:23.715736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:23.715736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:23.715736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:23.715736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:23.715736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:23.715736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:23.715736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:23.715736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:23.715736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:23.715736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:23.715736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:23.715736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:23.715736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:23.715736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:23.715736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:23.715736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:23.716518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:23.716518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:23.716518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:23.716518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:23.716518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:23.716518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:23.716518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:23.716518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:23.716518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:23.716518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:23.716518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 12:30:23.716518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:30:23.717498 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 143/281] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:30:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:30:23.725755 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a643842-9af1-4d28-a85a-17ef33232903 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:23.774200 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:23.808300 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530a63e-fe79-451b-a891-3c0df3cae9f5 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:23.857271 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67523ca5-6d69-4c6e-8d39-21f7b63e7235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:23.857834 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67523ca5-6d69-4c6e-8d39-21f7b63e7235 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:23.858117 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67523ca5-6d69-4c6e-8d39-21f7b63e7235 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:23.858387 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67523ca5-6d69-4c6e-8d39-21f7b63e7235 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:23.859120 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67523ca5-6d69-4c6e-8d39-21f7b63e7235 None admin] RBAC: Adding query filter params (role.id=47d5e2ea167241d5a27c8bbf4593eead, user.id=f9b08b3f0ce14cd5ae6f436921a45b9c, scope.system=all) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:23.860021 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67523ca5-6d69-4c6e-8d39-21f7b63e7235 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:23.861359 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67523ca5-6d69-4c6e-8d39-21f7b63e7235 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:23.867653 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 139/282] 10.222.0.33 () {56 vars in 1308 bytes} [Mon Oct 3 12:30:23 2022] GET /identity/v3/role_assignments?user.id=f9b08b3f0ce14cd5ae6f436921a45b9c&scope.system=all&role.id=47d5e2ea167241d5a27c8bbf4593eead => generated 226 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:23.877496 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a643842-9af1-4d28-a85a-17ef33232903 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:23.883999 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b38213-58fe-403b-80ec-3dbbdffda026 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:23.884296 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b38213-58fe-403b-80ec-3dbbdffda026 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:23.884543 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b38213-58fe-403b-80ec-3dbbdffda026 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:23.887560 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 140/283] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 12:30:23.892253 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce041c-3adb-4879-89d9-b2f7773df16b None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:23.892755 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce041c-3adb-4879-89d9-b2f7773df16b None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:23.893215 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce041c-3adb-4879-89d9-b2f7773df16b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:23.918881 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-71ce041c-3adb-4879-89d9-b2f7773df16b None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:23.920696 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-71ce041c-3adb-4879-89d9-b2f7773df16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:23.944515 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96224e64-bec8-4d4a-9a39-a136418ffc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:23.944702 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96224e64-bec8-4d4a-9a39-a136418ffc38 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:23.944927 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96224e64-bec8-4d4a-9a39-a136418ffc38 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:23.945120 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96224e64-bec8-4d4a-9a39-a136418ffc38 None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:23.945794 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96224e64-bec8-4d4a-9a39-a136418ffc38 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:23.946133 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96224e64-bec8-4d4a-9a39-a136418ffc38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:23.948268 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96224e64-bec8-4d4a-9a39-a136418ffc38 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:23.955202 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:23.960463 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 144/284] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 12:30:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:23.967988 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96224e64-bec8-4d4a-9a39-a136418ffc38 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:24.029494 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 141/285] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 12:30:24.137432 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96224e64-bec8-4d4a-9a39-a136418ffc38 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:24.249023 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3acfa8-16de-4e1d-9b08-d02ac06b59dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:24.249694 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3acfa8-16de-4e1d-9b08-d02ac06b59dd None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:24.249973 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3acfa8-16de-4e1d-9b08-d02ac06b59dd None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:24.250243 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3acfa8-16de-4e1d-9b08-d02ac06b59dd None admin] PATH_INFO: `/v3/users/74a000af0aa741488d5a21ee9d974ea5` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:24.252169 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3acfa8-16de-4e1d-9b08-d02ac06b59dd None admin] RBAC: Authorizing `identity:get_user(user_id=74a000af0aa741488d5a21ee9d974ea5)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:24.256145 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3acfa8-16de-4e1d-9b08-d02ac06b59dd None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:24.256145 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 145/286] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:30:23 2022] GET /identity/v3/users/74a000af0aa741488d5a21ee9d974ea5 => generated 297 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:24.264668 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce041c-3adb-4879-89d9-b2f7773df16b None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:24.320849 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3acfa8-16de-4e1d-9b08-d02ac06b59dd None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:24.428245 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:24.435800 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:24.456152 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce041c-3adb-4879-89d9-b2f7773df16b None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:24.486579 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3acfa8-16de-4e1d-9b08-d02ac06b59dd None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:24.532732 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa42d489-f455-4025-bc8b-ba0a4b5b563e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:24.533652 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa42d489-f455-4025-bc8b-ba0a4b5b563e None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:24.534109 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa42d489-f455-4025-bc8b-ba0a4b5b563e None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:24.534553 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa42d489-f455-4025-bc8b-ba0a4b5b563e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:24.541341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cdc718-c001-4db9-a58c-2af103e046e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:24.542067 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cdc718-c001-4db9-a58c-2af103e046e6 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:24.542513 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cdc718-c001-4db9-a58c-2af103e046e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:24.542855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cdc718-c001-4db9-a58c-2af103e046e6 None admin] PATH_INFO: `/v3/projects/c974a924a68947af9e8abc10c2614ae7` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:24.547127 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa42d489-f455-4025-bc8b-ba0a4b5b563e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:24.548103 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cdc718-c001-4db9-a58c-2af103e046e6 None admin] RBAC: Authorizing `identity:get_project(project_id=c974a924a68947af9e8abc10c2614ae7)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:24.549639 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cdc718-c001-4db9-a58c-2af103e046e6 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:24.553543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-aa42d489-f455-4025-bc8b-ba0a4b5b563e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 12:30:24.553543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:24.553543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:24.553543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:24.553543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:24.553543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:24.553543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:24.553543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:24.553543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:24.553543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:24.553543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:24.553543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:24.553543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:24.553543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:24.553543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:24.553543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:24.554400 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:24.554400 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:24.554400 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:24.554400 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:24.554400 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:24.554400 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:24.554400 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:24.554400 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:24.554400 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:24.554400 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:24.554400 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:24.554400 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:24.554400 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:24.554400 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:24.554400 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:24.554400 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:24.555172 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:24.555172 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:24.555172 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:24.555172 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:24.555172 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:24.555172 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:24.555172 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:24.555172 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:24.555172 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:24.555172 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:24.555172 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 12:30:24.555172 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:30:24.555963 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 142/287] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:30:24 2022] GET /identity/v3/projects/c974a924a68947af9e8abc10c2614ae7 => generated 302 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:24.558046 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 146/288] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:30:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:30:24.563642 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cdc718-c001-4db9-a58c-2af103e046e6 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:24.564726 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa42d489-f455-4025-bc8b-ba0a4b5b563e None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:24.714706 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa42d489-f455-4025-bc8b-ba0a4b5b563e None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:24.747488 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:24.757960 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482023de-81ef-4b07-a713-87e6c8e2ae49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:24.763876 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482023de-81ef-4b07-a713-87e6c8e2ae49 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:24.763876 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482023de-81ef-4b07-a713-87e6c8e2ae49 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:24.763876 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482023de-81ef-4b07-a713-87e6c8e2ae49 None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:24.763876 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482023de-81ef-4b07-a713-87e6c8e2ae49 None admin] RBAC: Adding query filter params (name=member) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:24.763876 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482023de-81ef-4b07-a713-87e6c8e2ae49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:24.763876 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482023de-81ef-4b07-a713-87e6c8e2ae49 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:24.772681 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 147/289] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 12:30:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:30:24.787369 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482023de-81ef-4b07-a713-87e6c8e2ae49 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:24.881715 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cdc718-c001-4db9-a58c-2af103e046e6 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:25.027604 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482023de-81ef-4b07-a713-87e6c8e2ae49 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:25.058137 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f66862-96b1-4ffe-82fa-ea43195605c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:25.058747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f66862-96b1-4ffe-82fa-ea43195605c1 None admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:25.059026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f66862-96b1-4ffe-82fa-ea43195605c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:25.059305 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f66862-96b1-4ffe-82fa-ea43195605c1 None admin] PATH_INFO: `/v3/projects/c974a924a68947af9e8abc10c2614ae7/users/74a000af0aa741488d5a21ee9d974ea5/roles/5c094fe41dc347cbbb61cb9342eee721` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:25.069165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f66862-96b1-4ffe-82fa-ea43195605c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=c974a924a68947af9e8abc10c2614ae7, user_id=74a000af0aa741488d5a21ee9d974ea5, role_id=5c094fe41dc347cbbb61cb9342eee721)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:25.070508 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f66862-96b1-4ffe-82fa-ea43195605c1 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:25.192040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3283d269-3c7d-4ef3-b221-92abd9a5d0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:25.192739 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283d269-3c7d-4ef3-b221-92abd9a5d0fe None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:25.193945 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283d269-3c7d-4ef3-b221-92abd9a5d0fe None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:25.194296 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283d269-3c7d-4ef3-b221-92abd9a5d0fe None admin] PATH_INFO: `/v3/users/362c446f51cd4a3e92e8721fa400383b` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:25.200060 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3283d269-3c7d-4ef3-b221-92abd9a5d0fe None admin] RBAC: Authorizing `identity:get_user(user_id=362c446f51cd4a3e92e8721fa400383b)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:25.201524 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3283d269-3c7d-4ef3-b221-92abd9a5d0fe None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:25.203133 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 148/290] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:30:24 2022] GET /identity/v3/users/362c446f51cd4a3e92e8721fa400383b => generated 297 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:25.208552 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3283d269-3c7d-4ef3-b221-92abd9a5d0fe None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:25.278644 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 143/291] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 12:30:24 2022] PUT /identity/v3/projects/c974a924a68947af9e8abc10c2614ae7/users/74a000af0aa741488d5a21ee9d974ea5/roles/5c094fe41dc347cbbb61cb9342eee721 => generated 0 bytes in 717 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:30:25.401056 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:25.459429 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3283d269-3c7d-4ef3-b221-92abd9a5d0fe None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:25.567342 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ead07f-e46e-422d-b722-af4f38ccee7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:25.567342 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ead07f-e46e-422d-b722-af4f38ccee7a None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:25.567342 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ead07f-e46e-422d-b722-af4f38ccee7a None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:25.567342 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ead07f-e46e-422d-b722-af4f38ccee7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:25.679432 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ead07f-e46e-422d-b722-af4f38ccee7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:25.679432 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ead07f-e46e-422d-b722-af4f38ccee7a None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:25.807984 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:25.950520 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 149/292] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:30:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 742 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 12:30:25.956515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f66862-96b1-4ffe-82fa-ea43195605c1 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:25.964368 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:26.239439 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f66862-96b1-4ffe-82fa-ea43195605c1 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:26.316981 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a05ae12-2ae0-4072-a5de-3240583e8793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:26.317631 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05ae12-2ae0-4072-a5de-3240583e8793 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:26.317911 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05ae12-2ae0-4072-a5de-3240583e8793 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:26.318280 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05ae12-2ae0-4072-a5de-3240583e8793 None admin] PATH_INFO: `/v3/projects/c974a924a68947af9e8abc10c2614ae7` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:26.324253 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a05ae12-2ae0-4072-a5de-3240583e8793 None admin] RBAC: Authorizing `identity:get_project(project_id=c974a924a68947af9e8abc10c2614ae7)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:26.325322 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a05ae12-2ae0-4072-a5de-3240583e8793 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:26.327030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 144/293] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:30:25 2022] GET /identity/v3/projects/c974a924a68947af9e8abc10c2614ae7 => generated 302 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:26.339947 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ead07f-e46e-422d-b722-af4f38ccee7a None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:26.527430 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ead07f-e46e-422d-b722-af4f38ccee7a None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:26.596762 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14fab9-2548-4763-a13c-45843fddd7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:26.596962 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762c810-3cc0-4285-aa64-10305ebd0efc None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:26.597294 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762c810-3cc0-4285-aa64-10305ebd0efc None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:26.597571 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762c810-3cc0-4285-aa64-10305ebd0efc None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:26.598478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14fab9-2548-4763-a13c-45843fddd7c6 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:26.598938 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 145/294] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:26.599683 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14fab9-2548-4763-a13c-45843fddd7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:26.600314 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14fab9-2548-4763-a13c-45843fddd7c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:26.601252 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac14fab9-2548-4763-a13c-45843fddd7c6 None admin] RBAC: Adding query filter params (role.id=47d5e2ea167241d5a27c8bbf4593eead, scope.project.id=c974a924a68947af9e8abc10c2614ae7, user.id=362c446f51cd4a3e92e8721fa400383b) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:26.601824 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac14fab9-2548-4763-a13c-45843fddd7c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:26.611583 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac14fab9-2548-4763-a13c-45843fddd7c6 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:26.612757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1214e1b-3aa1-4dcf-a18a-1ceaabc87160 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:26.613043 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1214e1b-3aa1-4dcf-a18a-1ceaabc87160 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:26.613318 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1214e1b-3aa1-4dcf-a18a-1ceaabc87160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:26.639397 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 150/295] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:30:26 2022] GET /identity/v3/role_assignments?user.id=362c446f51cd4a3e92e8721fa400383b&scope.project.id=c974a924a68947af9e8abc10c2614ae7&role.id=47d5e2ea167241d5a27c8bbf4593eead => generated 608 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:26.660552 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-c1214e1b-3aa1-4dcf-a18a-1ceaabc87160 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:26.662421 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-c1214e1b-3aa1-4dcf-a18a-1ceaabc87160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:26.689967 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9d2ff-f43d-40d1-8f8b-4b644c1bc81f None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:26.690260 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9d2ff-f43d-40d1-8f8b-4b644c1bc81f None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:26.690531 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9d2ff-f43d-40d1-8f8b-4b644c1bc81f None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:26.698067 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 151/296] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 12:30:26.700179 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44394a60-59db-4ced-9037-d2b08f79d9b5 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:26.700824 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44394a60-59db-4ced-9037-d2b08f79d9b5 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:26.700824 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44394a60-59db-4ced-9037-d2b08f79d9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:26.726409 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-44394a60-59db-4ced-9037-d2b08f79d9b5 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:26.726409 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-44394a60-59db-4ced-9037-d2b08f79d9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:26.743161 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:26.862921 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 152/297] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:30:26.864160 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 146/298] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:30:26.868397 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a9b67-e1c2-4928-8167-a49469da54a6 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:26.868397 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a9b67-e1c2-4928-8167-a49469da54a6 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:26.868397 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a9b67-e1c2-4928-8167-a49469da54a6 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:26.868397 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 153/299] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:26.876994 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fc4f1-25b9-452e-a1d4-5d37cfd5c313 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:26.877499 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fc4f1-25b9-452e-a1d4-5d37cfd5c313 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:26.877945 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fc4f1-25b9-452e-a1d4-5d37cfd5c313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:26.886668 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:26.905789 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-1e0fc4f1-25b9-452e-a1d4-5d37cfd5c313 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:26.908948 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1e0fc4f1-25b9-452e-a1d4-5d37cfd5c313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:27.009321 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 147/300] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 12:30:27.163850 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6a9b67-e1c2-4928-8167-a49469da54a6 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:27.286723 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0fc4f1-25b9-452e-a1d4-5d37cfd5c313 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:27.297389 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6a9b67-e1c2-4928-8167-a49469da54a6 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:27.360787 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d405d8e-55f3-46df-a1ff-b8a64a6e4d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:27.363269 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d405d8e-55f3-46df-a1ff-b8a64a6e4d97 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:27.363570 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d405d8e-55f3-46df-a1ff-b8a64a6e4d97 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:27.363840 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d405d8e-55f3-46df-a1ff-b8a64a6e4d97 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:27.400738 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d405d8e-55f3-46df-a1ff-b8a64a6e4d97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:27.416647 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-3d405d8e-55f3-46df-a1ff-b8a64a6e4d97 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 12:30:27.416647 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:27.416647 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:27.416647 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:27.416647 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:27.416647 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:27.416647 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:27.416647 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:27.416647 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:27.416647 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:27.416647 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:27.416647 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:27.416647 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:27.416647 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:27.416647 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:27.416647 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:27.417634 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:27.417634 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:27.417634 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:27.417634 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:27.417634 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:27.417634 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:27.417634 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:27.417634 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:27.417634 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:27.417634 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:27.417634 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:27.417634 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:27.417634 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:27.417634 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:27.417634 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:27.417634 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:27.418763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:27.418763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:27.418763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:27.418763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:27.418763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:27.418763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:27.418763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:27.418763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:27.418763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:27.418763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:27.418763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 12:30:27.418763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:30:27.418763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 154/301] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:30:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:30:27.418763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d405d8e-55f3-46df-a1ff-b8a64a6e4d97 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:27.439993 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:27.498117 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0fc4f1-25b9-452e-a1d4-5d37cfd5c313 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:27.545437 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d405d8e-55f3-46df-a1ff-b8a64a6e4d97 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:27.556139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:27.563575 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a743c-ccdd-4642-a57f-3c4a16e4c434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:27.564224 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a743c-ccdd-4642-a57f-3c4a16e4c434 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:27.567060 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a743c-ccdd-4642-a57f-3c4a16e4c434 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:27.567060 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a743c-ccdd-4642-a57f-3c4a16e4c434 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:27.567060 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a743c-ccdd-4642-a57f-3c4a16e4c434 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:27.567455 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a743c-ccdd-4642-a57f-3c4a16e4c434 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:27.570642 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 148/302] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:30:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:30:27.571343 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a743c-ccdd-4642-a57f-3c4a16e4c434 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:27.602090 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52badcc2-170b-4e14-9a41-5003adcd886c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:27.603055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52badcc2-170b-4e14-9a41-5003adcd886c None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:27.603358 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52badcc2-170b-4e14-9a41-5003adcd886c None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:27.603647 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52badcc2-170b-4e14-9a41-5003adcd886c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:27.619612 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52badcc2-170b-4e14-9a41-5003adcd886c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:27.626036 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-52badcc2-170b-4e14-9a41-5003adcd886c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 12:30:27.626036 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:27.626036 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:27.626036 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:27.626036 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:27.626036 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:27.626036 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:27.626036 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:27.626036 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:27.626036 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:27.626036 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:27.626036 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:27.626036 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:27.626036 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:27.626036 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:27.626036 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:27.627089 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:27.627089 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:27.627089 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:27.627089 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:27.627089 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:27.627089 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:27.627089 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:27.627089 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:27.627089 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:27.627089 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:27.627089 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:27.627089 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:27.627089 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:27.627089 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:27.627089 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:27.627089 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:27.628102 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:27.628102 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:27.628102 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:27.628102 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:27.628102 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:27.628102 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:27.628102 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:27.628102 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:27.628102 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:27.628102 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:27.628102 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 12:30:27.628102 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:30:27.635762 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 155/303] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:30:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:30:27.636259 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52badcc2-170b-4e14-9a41-5003adcd886c None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:27.702193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a743c-ccdd-4642-a57f-3c4a16e4c434 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:27.709420 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:27.740583 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec853cb-25fa-4525-8a65-7abb947d0df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:27.741199 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec853cb-25fa-4525-8a65-7abb947d0df9 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:27.741535 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec853cb-25fa-4525-8a65-7abb947d0df9 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:27.741772 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52badcc2-170b-4e14-9a41-5003adcd886c None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:27.741893 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec853cb-25fa-4525-8a65-7abb947d0df9 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:27.742449 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec853cb-25fa-4525-8a65-7abb947d0df9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:27.742794 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec853cb-25fa-4525-8a65-7abb947d0df9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:27.745208 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec853cb-25fa-4525-8a65-7abb947d0df9 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:27.761012 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 149/304] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 12:30:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:27.761012 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec853cb-25fa-4525-8a65-7abb947d0df9 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:27.775966 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb40b7-e64d-4f44-9bea-92ad2d2d9e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:27.776586 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb40b7-e64d-4f44-9bea-92ad2d2d9e5d None admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:27.776869 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb40b7-e64d-4f44-9bea-92ad2d2d9e5d None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:27.777143 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb40b7-e64d-4f44-9bea-92ad2d2d9e5d None admin] PATH_INFO: `/v3/groups` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:27.777768 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fb40b7-e64d-4f44-9bea-92ad2d2d9e5d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:27.779209 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fb40b7-e64d-4f44-9bea-92ad2d2d9e5d None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:27.829116 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:27.849743 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 156/305] 10.222.0.33 () {60 vars in 1134 bytes} [Mon Oct 3 12:30:27 2022] POST /identity/v3/groups => generated 229 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:30:27.850951 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb40b7-e64d-4f44-9bea-92ad2d2d9e5d None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:27.883856 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec853cb-25fa-4525-8a65-7abb947d0df9 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:27.933626 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e1355-dd55-4b71-a2ee-bfcd997f148a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:27.934612 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e1355-dd55-4b71-a2ee-bfcd997f148a None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:27.935088 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e1355-dd55-4b71-a2ee-bfcd997f148a None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:27.935764 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e1355-dd55-4b71-a2ee-bfcd997f148a None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:27.936759 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7e1355-dd55-4b71-a2ee-bfcd997f148a None admin] RBAC: Adding query filter params (name=member) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:27.937249 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7e1355-dd55-4b71-a2ee-bfcd997f148a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:27.939049 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7e1355-dd55-4b71-a2ee-bfcd997f148a None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:27.950915 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 150/306] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 12:30:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:27.961149 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e1355-dd55-4b71-a2ee-bfcd997f148a None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:28.009353 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb40b7-e64d-4f44-9bea-92ad2d2d9e5d None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:28.044650 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:28.051972 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35944494-ab44-4d7f-904f-61a129504089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:28.053168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35944494-ab44-4d7f-904f-61a129504089 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:28.053626 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35944494-ab44-4d7f-904f-61a129504089 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:28.053849 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35944494-ab44-4d7f-904f-61a129504089 None admin] PATH_INFO: `/v3/users/d0692f0f27574133a16641b99a131804` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:28.071579 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35944494-ab44-4d7f-904f-61a129504089 None admin] RBAC: Authorizing `identity:get_user(user_id=d0692f0f27574133a16641b99a131804)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:28.073044 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35944494-ab44-4d7f-904f-61a129504089 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:28.074707 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 157/307] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:30:27 2022] GET /identity/v3/users/d0692f0f27574133a16641b99a131804 => generated 293 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:28.075971 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e91ca-e5a8-4326-856d-836525bc28c0 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:28.076252 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e91ca-e5a8-4326-856d-836525bc28c0 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:28.076522 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e91ca-e5a8-4326-856d-836525bc28c0 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:28.077747 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 158/308] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:28.080487 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e1355-dd55-4b71-a2ee-bfcd997f148a None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:28.081754 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e91ca-e5a8-4326-856d-836525bc28c0 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:28.108277 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:28.124969 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1eead-cc26-4e37-b731-f82bec654895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:28.125667 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1eead-cc26-4e37-b731-f82bec654895 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:28.125942 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1eead-cc26-4e37-b731-f82bec654895 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:28.126216 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1eead-cc26-4e37-b731-f82bec654895 None admin] PATH_INFO: `/v3/users/f9b08b3f0ce14cd5ae6f436921a45b9c` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:28.127388 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b1eead-cc26-4e37-b731-f82bec654895 None admin] RBAC: Authorizing `identity:get_user(user_id=f9b08b3f0ce14cd5ae6f436921a45b9c)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:28.130618 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b1eead-cc26-4e37-b731-f82bec654895 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:28.136817 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 151/309] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:30:27 2022] GET /identity/v3/users/f9b08b3f0ce14cd5ae6f436921a45b9c => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:28.136817 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf165058-2cdd-46ce-a027-4c2f257ff7bb None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:28.136817 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf165058-2cdd-46ce-a027-4c2f257ff7bb None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:28.136817 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf165058-2cdd-46ce-a027-4c2f257ff7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:28.155519 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-cf165058-2cdd-46ce-a027-4c2f257ff7bb None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:28.157185 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-cf165058-2cdd-46ce-a027-4c2f257ff7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:28.192125 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e91ca-e5a8-4326-856d-836525bc28c0 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:28.231543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6967e95-9e98-4f55-b38d-c081f5cf2288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:28.232075 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6967e95-9e98-4f55-b38d-c081f5cf2288 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:28.232356 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6967e95-9e98-4f55-b38d-c081f5cf2288 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:28.232627 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6967e95-9e98-4f55-b38d-c081f5cf2288 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:28.233378 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6967e95-9e98-4f55-b38d-c081f5cf2288 None admin] RBAC: Adding query filter params (role.id=5c094fe41dc347cbbb61cb9342eee721, user.id=d0692f0f27574133a16641b99a131804, scope.system=all) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:28.233681 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6967e95-9e98-4f55-b38d-c081f5cf2288 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:28.235352 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6967e95-9e98-4f55-b38d-c081f5cf2288 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:28.237811 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 152/310] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:30:28.238481 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf165058-2cdd-46ce-a027-4c2f257ff7bb None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:28.245818 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 159/311] 10.222.0.33 () {56 vars in 1308 bytes} [Mon Oct 3 12:30:28 2022] GET /identity/v3/role_assignments?user.id=d0692f0f27574133a16641b99a131804&scope.system=all&role.id=5c094fe41dc347cbbb61cb9342eee721 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:30:28.363455 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf165058-2cdd-46ce-a027-4c2f257ff7bb None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:28.391611 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:28.404718 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d364df7b-d9b5-4340-81e1-ba6fbb5329ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:28.405462 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364df7b-d9b5-4340-81e1-ba6fbb5329ff None admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:28.406963 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364df7b-d9b5-4340-81e1-ba6fbb5329ff None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:28.406963 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364df7b-d9b5-4340-81e1-ba6fbb5329ff None admin] PATH_INFO: `/v3/system/users/f9b08b3f0ce14cd5ae6f436921a45b9c/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:28.407600 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d364df7b-d9b5-4340-81e1-ba6fbb5329ff None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f9b08b3f0ce14cd5ae6f436921a45b9c, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:28.409016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d364df7b-d9b5-4340-81e1-ba6fbb5329ff None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:28.513144 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6967e95-9e98-4f55-b38d-c081f5cf2288 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:28.520287 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 153/312] 10.222.0.33 () {58 vars in 1334 bytes} [Mon Oct 3 12:30:28 2022] PUT /identity/v3/system/users/f9b08b3f0ce14cd5ae6f436921a45b9c/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:30:28.575309 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:28.631978 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6967e95-9e98-4f55-b38d-c081f5cf2288 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:28.670416 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7ea5b-ca44-4f46-93b4-a7bdd3a0e2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:28.671077 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7ea5b-ca44-4f46-93b4-a7bdd3a0e2f7 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:28.671386 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7ea5b-ca44-4f46-93b4-a7bdd3a0e2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:28.671786 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7ea5b-ca44-4f46-93b4-a7bdd3a0e2f7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:28.684712 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7ea5b-ca44-4f46-93b4-a7bdd3a0e2f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:28.690962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-0fa7ea5b-ca44-4f46-93b4-a7bdd3a0e2f7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 12:30:28.690962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:28.690962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:28.690962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:28.690962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:28.690962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:28.690962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:28.690962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:28.690962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:28.690962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:28.690962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:28.690962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:28.690962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:28.690962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:28.690962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:28.690962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:28.691923 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:28.691923 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:28.691923 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:28.691923 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:28.691923 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:28.691923 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:28.691923 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:28.691923 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:28.691923 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:28.691923 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:28.691923 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:28.691923 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:28.691923 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:28.691923 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:28.691923 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:28.691923 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:28.692906 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:28.692906 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:28.692906 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:28.692906 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:28.692906 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:28.692906 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:28.692906 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:28.692906 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:28.692906 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:28.692906 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:28.692906 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 12:30:28.692906 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:30:28.692906 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 160/313] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:30:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:30:28.698260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d364df7b-d9b5-4340-81e1-ba6fbb5329ff None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:28.822159 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d364df7b-d9b5-4340-81e1-ba6fbb5329ff None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:28.861088 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a53e2-a721-4e12-a077-46dd85a324fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:28.862077 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a53e2-a721-4e12-a077-46dd85a324fe None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:28.862548 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a53e2-a721-4e12-a077-46dd85a324fe None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:28.863073 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a53e2-a721-4e12-a077-46dd85a324fe None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:28.863945 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1a53e2-a721-4e12-a077-46dd85a324fe None admin] RBAC: Adding query filter params (name=reader) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:28.864457 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1a53e2-a721-4e12-a077-46dd85a324fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:28.866778 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1a53e2-a721-4e12-a077-46dd85a324fe None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:28.876534 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 154/314] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 12:30:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:28.885270 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7ea5b-ca44-4f46-93b4-a7bdd3a0e2f7 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:29.115881 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7ea5b-ca44-4f46-93b4-a7bdd3a0e2f7 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:29.129365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:29.151114 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f5cda-f636-4cb4-8595-3c40d9968d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:29.151787 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f5cda-f636-4cb4-8595-3c40d9968d46 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:29.152069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f5cda-f636-4cb4-8595-3c40d9968d46 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:29.158056 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f5cda-f636-4cb4-8595-3c40d9968d46 None admin] PATH_INFO: `/v3/users/74a000af0aa741488d5a21ee9d974ea5` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:29.158056 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f5cda-f636-4cb4-8595-3c40d9968d46 None admin] RBAC: Authorizing `identity:get_user(user_id=74a000af0aa741488d5a21ee9d974ea5)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:29.158056 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f5cda-f636-4cb4-8595-3c40d9968d46 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:29.158056 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 161/315] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:30:28 2022] GET /identity/v3/users/74a000af0aa741488d5a21ee9d974ea5 => generated 297 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:29.162757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a53e2-a721-4e12-a077-46dd85a324fe None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:29.219134 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:29.284569 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a53e2-a721-4e12-a077-46dd85a324fe None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:29.324268 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e315dc-8943-4f6c-9ad1-02214e30f64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:29.324702 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e315dc-8943-4f6c-9ad1-02214e30f64a None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:29.325076 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e315dc-8943-4f6c-9ad1-02214e30f64a None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:29.325405 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e315dc-8943-4f6c-9ad1-02214e30f64a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:29.403118 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e315dc-8943-4f6c-9ad1-02214e30f64a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:29.404595 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e315dc-8943-4f6c-9ad1-02214e30f64a None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:29.457297 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:29.510932 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 155/316] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:30:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 12:30:29.518030 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f5cda-f636-4cb4-8595-3c40d9968d46 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:29.634755 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f5cda-f636-4cb4-8595-3c40d9968d46 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:29.668931 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b21ade2-755f-417c-ba56-a276cd19206c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:29.669492 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21ade2-755f-417c-ba56-a276cd19206c None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:29.669770 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21ade2-755f-417c-ba56-a276cd19206c None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:29.670042 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21ade2-755f-417c-ba56-a276cd19206c None admin] PATH_INFO: `/v3/projects/c974a924a68947af9e8abc10c2614ae7` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:29.671188 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b21ade2-755f-417c-ba56-a276cd19206c None admin] RBAC: Authorizing `identity:get_project(project_id=c974a924a68947af9e8abc10c2614ae7)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:29.672652 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b21ade2-755f-417c-ba56-a276cd19206c None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:29.674274 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 162/317] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:30:29 2022] GET /identity/v3/projects/c974a924a68947af9e8abc10c2614ae7 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:29.684400 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e315dc-8943-4f6c-9ad1-02214e30f64a None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:29.795603 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e315dc-8943-4f6c-9ad1-02214e30f64a None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:29.833385 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc1fc9-bee9-4ab8-95f7-550409ddfb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:29.834100 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc1fc9-bee9-4ab8-95f7-550409ddfb2c None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:29.834525 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc1fc9-bee9-4ab8-95f7-550409ddfb2c None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:29.834813 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc1fc9-bee9-4ab8-95f7-550409ddfb2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:29.835589 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dc1fc9-bee9-4ab8-95f7-550409ddfb2c None admin] RBAC: Adding query filter params (role.id=5c094fe41dc347cbbb61cb9342eee721, scope.project.id=c974a924a68947af9e8abc10c2614ae7, user.id=74a000af0aa741488d5a21ee9d974ea5) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:29.835899 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dc1fc9-bee9-4ab8-95f7-550409ddfb2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:29.837507 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dc1fc9-bee9-4ab8-95f7-550409ddfb2c None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:29.844226 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 156/318] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:30:29 2022] GET /identity/v3/role_assignments?user.id=74a000af0aa741488d5a21ee9d974ea5&scope.project.id=c974a924a68947af9e8abc10c2614ae7&role.id=5c094fe41dc347cbbb61cb9342eee721 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:30.227580 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef30c46-d72d-4502-a12d-e82aada23038 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:30.227982 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef30c46-d72d-4502-a12d-e82aada23038 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:30.228320 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef30c46-d72d-4502-a12d-e82aada23038 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:30.229758 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 163/319] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:30.235126 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f6c9d-2f28-47e0-81c0-f81a23caf445 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:30.236065 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f6c9d-2f28-47e0-81c0-f81a23caf445 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:30.238597 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f6c9d-2f28-47e0-81c0-f81a23caf445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:30.266490 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-3a2f6c9d-2f28-47e0-81c0-f81a23caf445 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:30.267811 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-3a2f6c9d-2f28-47e0-81c0-f81a23caf445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:30.279842 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:30.343519 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 157/320] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:30:30.588072 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d3dffa-341d-48a7-9e7d-edbebc50fae4 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:30.588589 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d3dffa-341d-48a7-9e7d-edbebc50fae4 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:30.588700 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d3dffa-341d-48a7-9e7d-edbebc50fae4 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:30.590124 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 164/321] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:30.595260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a602e0-41b9-42fe-8751-bd060a5f5bec None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:30.595594 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a602e0-41b9-42fe-8751-bd060a5f5bec None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:30.595878 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a602e0-41b9-42fe-8751-bd060a5f5bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:30.615024 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-b7a602e0-41b9-42fe-8751-bd060a5f5bec None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:30.615500 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d3dffa-341d-48a7-9e7d-edbebc50fae4 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:30.616285 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b7a602e0-41b9-42fe-8751-bd060a5f5bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:30.669208 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:30.685011 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 158/322] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:30:30.727467 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d3dffa-341d-48a7-9e7d-edbebc50fae4 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:30.764334 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61223714-1459-48d7-8410-0cee432891ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:30.764967 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61223714-1459-48d7-8410-0cee432891ab None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:30.765243 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61223714-1459-48d7-8410-0cee432891ab None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:30.765536 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61223714-1459-48d7-8410-0cee432891ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:30.766611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61223714-1459-48d7-8410-0cee432891ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:30.768079 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61223714-1459-48d7-8410-0cee432891ab None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:30.769815 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 165/323] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:30:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:30:30.776160 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a602e0-41b9-42fe-8751-bd060a5f5bec None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:30.843918 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190c36c-1cec-4a0a-b3a8-f9ba42c7ea81 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:30.844227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190c36c-1cec-4a0a-b3a8-f9ba42c7ea81 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:30.844410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190c36c-1cec-4a0a-b3a8-f9ba42c7ea81 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:30.846788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 166/324] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 12:30:30.851894 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aeacaf-20b5-4ba5-8c00-7b72baabaf05 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:30.852252 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aeacaf-20b5-4ba5-8c00-7b72baabaf05 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:30.852501 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:30.852606 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aeacaf-20b5-4ba5-8c00-7b72baabaf05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:30.872592 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-54aeacaf-20b5-4ba5-8c00-7b72baabaf05 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:30.873757 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-54aeacaf-20b5-4ba5-8c00-7b72baabaf05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:30.888279 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a602e0-41b9-42fe-8751-bd060a5f5bec None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:30.928941 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806dd2c-5d09-487b-97f3-e744bf5ebf08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:30.929595 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806dd2c-5d09-487b-97f3-e744bf5ebf08 None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:30.930048 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806dd2c-5d09-487b-97f3-e744bf5ebf08 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:30.930329 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806dd2c-5d09-487b-97f3-e744bf5ebf08 None admin] PATH_INFO: `/v3/groups` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:30.931291 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7806dd2c-5d09-487b-97f3-e744bf5ebf08 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:30.932823 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7806dd2c-5d09-487b-97f3-e744bf5ebf08 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:30.948590 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 167/325] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 12:30:30.957868 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aeacaf-20b5-4ba5-8c00-7b72baabaf05 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:31.004023 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:31.059466 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aeacaf-20b5-4ba5-8c00-7b72baabaf05 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:31.078274 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 159/326] 10.222.0.33 () {60 vars in 1134 bytes} [Mon Oct 3 12:30:30 2022] POST /identity/v3/groups => generated 226 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:30:31.094783 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09923e7f-fcb8-4573-9b40-dc1f269380bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:31.095394 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09923e7f-fcb8-4573-9b40-dc1f269380bd None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:31.095681 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09923e7f-fcb8-4573-9b40-dc1f269380bd None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:31.095954 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09923e7f-fcb8-4573-9b40-dc1f269380bd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:31.109501 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09923e7f-fcb8-4573-9b40-dc1f269380bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:31.114678 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-09923e7f-fcb8-4573-9b40-dc1f269380bd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 12:30:31.114678 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:31.114678 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:31.114678 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:31.114678 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:31.114678 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:31.114678 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:31.114678 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:31.114678 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:31.114678 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:31.114678 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:31.114678 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:31.114678 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:31.114678 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:31.114678 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:31.114678 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:31.116436 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:31.116436 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:31.116436 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:31.116436 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:31.116436 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:31.116436 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:31.116436 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:31.116436 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:31.116436 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:31.116436 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:31.116436 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:31.116436 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:31.116436 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:31.116436 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:31.116436 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:31.116436 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:31.117611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:31.117611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:31.117611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:31.117611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:31.117611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:31.117611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:31.117611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:31.117611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:31.117611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:31.117611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:31.117611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 12:30:31.117611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:30:31.117611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 168/327] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:30:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:30:31.122748 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806dd2c-5d09-487b-97f3-e744bf5ebf08 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:31.224351 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09923e7f-fcb8-4573-9b40-dc1f269380bd None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:31.259753 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806dd2c-5d09-487b-97f3-e744bf5ebf08 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:31.295208 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:31.300708 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629baeb9-399c-4184-8ea4-fa64d8afd4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:31.301325 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629baeb9-399c-4184-8ea4-fa64d8afd4e0 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:31.302882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629baeb9-399c-4184-8ea4-fa64d8afd4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:31.302882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629baeb9-399c-4184-8ea4-fa64d8afd4e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:31.303172 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629baeb9-399c-4184-8ea4-fa64d8afd4e0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:31.304079 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629baeb9-399c-4184-8ea4-fa64d8afd4e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:31.311016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629baeb9-399c-4184-8ea4-fa64d8afd4e0 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:31.315041 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 160/328] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 12:30:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:31.321318 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629baeb9-399c-4184-8ea4-fa64d8afd4e0 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:31.349553 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09923e7f-fcb8-4573-9b40-dc1f269380bd None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:31.360819 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:31.396607 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62cf24-5ebd-44b6-b478-d4242c001d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:31.396607 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62cf24-5ebd-44b6-b478-d4242c001d61 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:31.396607 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62cf24-5ebd-44b6-b478-d4242c001d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:31.396607 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62cf24-5ebd-44b6-b478-d4242c001d61 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:31.409004 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62cf24-5ebd-44b6-b478-d4242c001d61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:31.416917 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-8c62cf24-5ebd-44b6-b478-d4242c001d61 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 12:30:31.416917 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:31.416917 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:31.416917 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:31.416917 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:31.416917 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:31.416917 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:31.416917 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:31.416917 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:31.416917 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:31.416917 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:31.416917 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:31.416917 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:31.416917 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:31.416917 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:31.416917 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:31.418128 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:31.418128 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:31.418128 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:31.418128 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:31.418128 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:31.418128 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:31.418128 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:31.418128 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:31.418128 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:31.418128 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:31.418128 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:31.418128 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:31.418128 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:31.418128 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:31.418128 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:31.418128 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:31.419203 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:31.419203 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:31.419203 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:31.419203 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:31.419203 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:31.419203 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:31.419203 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:31.419203 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:31.419203 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:31.419203 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:31.419203 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 12:30:31.419203 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:30:31.419203 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 169/329] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:30:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:30:31.423499 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62cf24-5ebd-44b6-b478-d4242c001d61 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:31.444236 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629baeb9-399c-4184-8ea4-fa64d8afd4e0 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:31.482060 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c64dfb-415e-4706-8c22-ae2289bbf050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:31.482705 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c64dfb-415e-4706-8c22-ae2289bbf050 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:31.482979 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c64dfb-415e-4706-8c22-ae2289bbf050 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:31.483250 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c64dfb-415e-4706-8c22-ae2289bbf050 None admin] PATH_INFO: `/v3/users/d0692f0f27574133a16641b99a131804` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:31.485416 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c64dfb-415e-4706-8c22-ae2289bbf050 None admin] RBAC: Authorizing `identity:get_user(user_id=d0692f0f27574133a16641b99a131804)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:31.486840 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c64dfb-415e-4706-8c22-ae2289bbf050 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:31.491342 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 161/330] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:30:31 2022] GET /identity/v3/users/d0692f0f27574133a16641b99a131804 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:30:31.494964 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c64dfb-415e-4706-8c22-ae2289bbf050 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:31.560814 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62cf24-5ebd-44b6-b478-d4242c001d61 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:31.586872 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:31.634286 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c4041-1ff5-43f6-be36-366a78c20e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:31.634935 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c4041-1ff5-43f6-be36-366a78c20e0e None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:31.635126 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c4041-1ff5-43f6-be36-366a78c20e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:31.635436 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c4041-1ff5-43f6-be36-366a78c20e0e None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:31.636137 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910c4041-1ff5-43f6-be36-366a78c20e0e None admin] RBAC: Adding query filter params (name=member) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:31.636699 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910c4041-1ff5-43f6-be36-366a78c20e0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:31.638197 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910c4041-1ff5-43f6-be36-366a78c20e0e None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:31.642072 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c64dfb-415e-4706-8c22-ae2289bbf050 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:31.650077 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 170/331] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 12:30:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:30:31.658798 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c4041-1ff5-43f6-be36-366a78c20e0e None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:31.703417 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:31.711987 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbd1dce-dbe0-46d2-81b4-87ca9169701f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:31.715340 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd1dce-dbe0-46d2-81b4-87ca9169701f None admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:31.715340 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd1dce-dbe0-46d2-81b4-87ca9169701f None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:31.715340 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd1dce-dbe0-46d2-81b4-87ca9169701f None admin] PATH_INFO: `/v3/system/users/d0692f0f27574133a16641b99a131804/roles/5c094fe41dc347cbbb61cb9342eee721` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:31.715340 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbd1dce-dbe0-46d2-81b4-87ca9169701f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d0692f0f27574133a16641b99a131804, role_id=5c094fe41dc347cbbb61cb9342eee721)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:31.719340 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbd1dce-dbe0-46d2-81b4-87ca9169701f None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:31.783353 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c4041-1ff5-43f6-be36-366a78c20e0e None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:31.783541 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 162/332] 10.222.0.33 () {58 vars in 1334 bytes} [Mon Oct 3 12:30:31 2022] PUT /identity/v3/system/users/d0692f0f27574133a16641b99a131804/roles/5c094fe41dc347cbbb61cb9342eee721 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:30:31.822531 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531577c-12fa-4c0e-a5df-2bf064acb19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:31.823196 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531577c-12fa-4c0e-a5df-2bf064acb19e None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:31.823653 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531577c-12fa-4c0e-a5df-2bf064acb19e None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:31.824115 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531577c-12fa-4c0e-a5df-2bf064acb19e None admin] PATH_INFO: `/v3/users/f9b08b3f0ce14cd5ae6f436921a45b9c` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:31.826154 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6531577c-12fa-4c0e-a5df-2bf064acb19e None admin] RBAC: Authorizing `identity:get_user(user_id=f9b08b3f0ce14cd5ae6f436921a45b9c)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:31.828173 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6531577c-12fa-4c0e-a5df-2bf064acb19e None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:31.835074 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 171/333] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:30:31 2022] GET /identity/v3/users/f9b08b3f0ce14cd5ae6f436921a45b9c => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:31.836584 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbd1dce-dbe0-46d2-81b4-87ca9169701f None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:31.978434 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbd1dce-dbe0-46d2-81b4-87ca9169701f None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:32.009160 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:32.023102 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f38f1c-4af1-45eb-bdf5-43d33c708254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:32.023766 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f38f1c-4af1-45eb-bdf5-43d33c708254 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:32.024136 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f38f1c-4af1-45eb-bdf5-43d33c708254 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:32.024418 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f38f1c-4af1-45eb-bdf5-43d33c708254 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:32.025293 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f38f1c-4af1-45eb-bdf5-43d33c708254 None admin] RBAC: Adding query filter params (role.id=47d5e2ea167241d5a27c8bbf4593eead, user.id=f9b08b3f0ce14cd5ae6f436921a45b9c, scope.system=all) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:32.025599 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f38f1c-4af1-45eb-bdf5-43d33c708254 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:32.027053 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f38f1c-4af1-45eb-bdf5-43d33c708254 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:32.042836 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 163/334] 10.222.0.33 () {56 vars in 1308 bytes} [Mon Oct 3 12:30:31 2022] GET /identity/v3/role_assignments?user.id=f9b08b3f0ce14cd5ae6f436921a45b9c&scope.system=all&role.id=47d5e2ea167241d5a27c8bbf4593eead => generated 510 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:34.021813 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f1871-e195-4d66-a638-6acd39678cac None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:34.021813 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f1871-e195-4d66-a638-6acd39678cac None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:34.021813 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f1871-e195-4d66-a638-6acd39678cac None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:34.023064 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 172/335] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:34.037340 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e3fc3-16a3-4165-a55a-aa90d7e642eb None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:34.038281 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e3fc3-16a3-4165-a55a-aa90d7e642eb None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:34.038743 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e3fc3-16a3-4165-a55a-aa90d7e642eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:34.072434 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-009e3fc3-16a3-4165-a55a-aa90d7e642eb None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:34.073999 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-009e3fc3-16a3-4165-a55a-aa90d7e642eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:34.128225 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737da96-4d2c-4fd3-bdfa-3745575757b0 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:34.128849 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737da96-4d2c-4fd3-bdfa-3745575757b0 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:34.129317 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737da96-4d2c-4fd3-bdfa-3745575757b0 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:34.131735 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 173/336] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 12:30:34.148007 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd693f4-e448-4f9f-a052-bd1760bb8f26 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:34.148298 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd693f4-e448-4f9f-a052-bd1760bb8f26 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:34.148565 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd693f4-e448-4f9f-a052-bd1760bb8f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:34.182858 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 164/337] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:30:34.216624 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-0fd693f4-e448-4f9f-a052-bd1760bb8f26 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:34.217851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-0fd693f4-e448-4f9f-a052-bd1760bb8f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:34.321983 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 174/338] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 12:30:34.516604 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e3fc3-16a3-4165-a55a-aa90d7e642eb None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:34.643662 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e3fc3-16a3-4165-a55a-aa90d7e642eb None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:34.668311 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:34.683599 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f37ddb-d514-4147-88fc-e3abf4d1ad7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:34.684550 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f37ddb-d514-4147-88fc-e3abf4d1ad7c None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:34.685021 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f37ddb-d514-4147-88fc-e3abf4d1ad7c None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:34.685485 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f37ddb-d514-4147-88fc-e3abf4d1ad7c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:34.696890 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f37ddb-d514-4147-88fc-e3abf4d1ad7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:34.702518 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-a0f37ddb-d514-4147-88fc-e3abf4d1ad7c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 12:30:34.702518 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:34.702518 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:34.702518 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:34.702518 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:34.702518 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:34.702518 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:34.702518 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:34.702518 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:34.702518 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:34.702518 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:34.702518 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:34.702518 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:34.702518 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:34.702518 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:34.702518 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:34.703423 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:34.703423 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:34.703423 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:34.703423 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:34.703423 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:34.703423 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:34.703423 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:34.703423 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:34.703423 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:34.703423 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:34.703423 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:34.703423 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:34.703423 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:34.703423 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:34.703423 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:34.703423 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:34.704260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:34.704260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:34.704260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:34.704260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:34.704260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:34.704260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:34.704260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:34.704260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:34.704260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:34.704260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:34.704260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 12:30:34.704260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:30:34.711471 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 165/339] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:30:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:30:34.713007 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd693f4-e448-4f9f-a052-bd1760bb8f26 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:34.789000 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:34.938335 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd693f4-e448-4f9f-a052-bd1760bb8f26 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:34.941772 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f37ddb-d514-4147-88fc-e3abf4d1ad7c None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:35.044081 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13875a77-80c6-443b-9ffe-66143d60bc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:35.044716 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13875a77-80c6-443b-9ffe-66143d60bc68 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:35.044998 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13875a77-80c6-443b-9ffe-66143d60bc68 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:35.045273 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13875a77-80c6-443b-9ffe-66143d60bc68 None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:35.045949 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13875a77-80c6-443b-9ffe-66143d60bc68 None admin] RBAC: Adding query filter params (name=member) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:35.046251 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13875a77-80c6-443b-9ffe-66143d60bc68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:35.047671 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13875a77-80c6-443b-9ffe-66143d60bc68 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:35.062183 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 175/340] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 12:30:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:35.071751 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc270194-711e-4283-a001-d772563caeef None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:35.072295 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc270194-711e-4283-a001-d772563caeef None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:35.072726 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc270194-711e-4283-a001-d772563caeef None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:35.074516 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 176/341] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:35 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:35.076243 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc270194-711e-4283-a001-d772563caeef None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:35.154830 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f37ddb-d514-4147-88fc-e3abf4d1ad7c None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:35.191774 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:35.201964 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb71694a-409b-4c99-86af-de05c44e3c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:35.202718 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb71694a-409b-4c99-86af-de05c44e3c60 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:35.203071 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb71694a-409b-4c99-86af-de05c44e3c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:35.203510 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb71694a-409b-4c99-86af-de05c44e3c60 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:35.205304 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc270194-711e-4283-a001-d772563caeef None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:35.219789 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb71694a-409b-4c99-86af-de05c44e3c60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:35.225577 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-cb71694a-409b-4c99-86af-de05c44e3c60 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 12:30:35.225577 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:35.225577 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:35.225577 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:35.225577 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:35.225577 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:35.225577 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:35.225577 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:35.225577 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:35.225577 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:35.225577 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:35.225577 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:35.225577 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:35.225577 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:35.225577 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:35.225577 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:35.227036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:35.227036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:35.227036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:35.227036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:35.227036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:35.227036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:35.227036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:35.227036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:35.227036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:35.227036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:35.227036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:35.227036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:35.227036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:35.227036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:35.227036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:35.227036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:35.228299 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:35.228299 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:35.228299 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:35.228299 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:35.228299 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:35.228299 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:35.228299 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:35.228299 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:35.228299 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:35.228299 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:35.228299 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 12:30:35.228299 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:30:35.228299 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 166/342] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 12:30:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:30:35.229568 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6cc51-2405-4990-a5f5-e089d8568637 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:35.229948 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6cc51-2405-4990-a5f5-e089d8568637 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:35.230296 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6cc51-2405-4990-a5f5-e089d8568637 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:35.231935 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 167/343] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:35.233498 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce1e42-bfc0-4563-a698-01940a117965 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:35.233498 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce1e42-bfc0-4563-a698-01940a117965 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:35.233687 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce1e42-bfc0-4563-a698-01940a117965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:35.260786 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-91ce1e42-bfc0-4563-a698-01940a117965 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:35.261982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-91ce1e42-bfc0-4563-a698-01940a117965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:35.263979 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6effa5eb-e11e-4291-9a74-15a64e5424c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:35.264855 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6effa5eb-e11e-4291-9a74-15a64e5424c8 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:35.265327 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6effa5eb-e11e-4291-9a74-15a64e5424c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:35.265785 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6effa5eb-e11e-4291-9a74-15a64e5424c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:35.301257 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:35.345391 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6effa5eb-e11e-4291-9a74-15a64e5424c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:35.347396 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6effa5eb-e11e-4291-9a74-15a64e5424c8 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:35.350049 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 168/344] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:30:35.351363 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce1e42-bfc0-4563-a698-01940a117965 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:35.467402 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce1e42-bfc0-4563-a698-01940a117965 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:35.520986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:35.526989 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa038bb-ddd9-4f8b-880b-2d000e810f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:35.528015 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa038bb-ddd9-4f8b-880b-2d000e810f11 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:35.528437 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa038bb-ddd9-4f8b-880b-2d000e810f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:35.528845 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa038bb-ddd9-4f8b-880b-2d000e810f11 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:35.529662 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa038bb-ddd9-4f8b-880b-2d000e810f11 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:35.530125 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa038bb-ddd9-4f8b-880b-2d000e810f11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:35.531747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa038bb-ddd9-4f8b-880b-2d000e810f11 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:35.552445 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 169/345] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:30:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:35.557705 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c9eeb-eb64-4962-a4a0-c66bd1e55967 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:35.558042 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c9eeb-eb64-4962-a4a0-c66bd1e55967 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:35.558332 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c9eeb-eb64-4962-a4a0-c66bd1e55967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:35.564704 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 177/346] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:30:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 12:30:35.565623 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6effa5eb-e11e-4291-9a74-15a64e5424c8 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:35.607304 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-ef9c9eeb-eb64-4962-a4a0-c66bd1e55967 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:35.608605 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ef9c9eeb-eb64-4962-a4a0-c66bd1e55967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:35.666377 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:35.753176 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 170/347] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:30:35.754116 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9c9eeb-eb64-4962-a4a0-c66bd1e55967 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:35.766279 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6effa5eb-e11e-4291-9a74-15a64e5424c8 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:35.821537 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cec2698-e13e-4716-99be-ed6ff338139d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:35.822158 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cec2698-e13e-4716-99be-ed6ff338139d None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:35.822436 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cec2698-e13e-4716-99be-ed6ff338139d None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:35.822706 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cec2698-e13e-4716-99be-ed6ff338139d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:35.898247 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9c9eeb-eb64-4962-a4a0-c66bd1e55967 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:35.903747 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:35.920273 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cec2698-e13e-4716-99be-ed6ff338139d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:35.922604 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cec2698-e13e-4716-99be-ed6ff338139d None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:35.949822 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86948d4-6388-41cf-ad86-4cf0319641c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:35.950477 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86948d4-6388-41cf-ad86-4cf0319641c7 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:35.950752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86948d4-6388-41cf-ad86-4cf0319641c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:35.951023 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86948d4-6388-41cf-ad86-4cf0319641c7 None admin] PATH_INFO: `/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:35.952190 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86948d4-6388-41cf-ad86-4cf0319641c7 None admin] RBAC: Authorizing `identity:get_project(project_id=6ee4b811a8dc4bc39e6ed0d4275b1782)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:35.953628 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86948d4-6388-41cf-ad86-4cf0319641c7 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:35.959272 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 171/348] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:30:35 2022] GET /identity/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:35.959272 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86948d4-6388-41cf-ad86-4cf0319641c7 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:35.992799 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:36.062526 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 178/349] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:30:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 12:30:36.064688 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e6e03-a35d-411c-ad9b-7bc613f242e0 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:36.064973 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e6e03-a35d-411c-ad9b-7bc613f242e0 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:36.065242 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e6e03-a35d-411c-ad9b-7bc613f242e0 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:36.066700 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 179/350] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:36.067188 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e6e03-a35d-411c-ad9b-7bc613f242e0 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:36.099259 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86948d4-6388-41cf-ad86-4cf0319641c7 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:36.145134 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe662426-9eea-46c1-b3bd-73e18cae6c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:36.145590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe662426-9eea-46c1-b3bd-73e18cae6c3d None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:36.145878 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe662426-9eea-46c1-b3bd-73e18cae6c3d None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:36.146242 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe662426-9eea-46c1-b3bd-73e18cae6c3d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:36.160585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe662426-9eea-46c1-b3bd-73e18cae6c3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:36.165682 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-fe662426-9eea-46c1-b3bd-73e18cae6c3d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 12:30:36.165682 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:36.165682 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:36.165682 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:36.165682 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:36.165682 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:36.165682 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:36.165682 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:36.165682 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:36.165682 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:36.165682 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:36.165682 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:36.165682 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:36.165682 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:36.165682 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:36.165682 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:36.166647 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:36.166647 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:36.166647 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:36.166647 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:36.166647 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:36.166647 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:36.166647 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:36.166647 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:36.166647 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:36.166647 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:36.166647 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:36.166647 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:36.166647 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:36.166647 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:36.166647 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:36.166647 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:36.167837 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:36.167837 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:36.167837 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:36.167837 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:36.167837 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:36.167837 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:36.167837 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:36.167837 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:36.167837 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:36.167837 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:36.167837 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 12:30:36.167837 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:30:36.167837 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 172/351] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:30:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:30:36.167837 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe662426-9eea-46c1-b3bd-73e18cae6c3d None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:36.196167 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:36.215244 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e6e03-a35d-411c-ad9b-7bc613f242e0 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:36.257495 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114f1bd-5801-400e-9447-77662702ce9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:36.258140 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114f1bd-5801-400e-9447-77662702ce9f None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:36.258423 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114f1bd-5801-400e-9447-77662702ce9f None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:36.258695 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114f1bd-5801-400e-9447-77662702ce9f None admin] PATH_INFO: `/v3/groups/93681966bd5a46368ddcce563e694a25` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:36.266866 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b114f1bd-5801-400e-9447-77662702ce9f None admin] RBAC: Authorizing `identity:get_group(group_id=93681966bd5a46368ddcce563e694a25)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:36.268335 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b114f1bd-5801-400e-9447-77662702ce9f None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:36.271656 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 180/352] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Oct 3 12:30:36 2022] GET /identity/v3/groups/93681966bd5a46368ddcce563e694a25 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:36.272547 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38730261-3655-436c-947f-15f9753245f2 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:36.272836 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38730261-3655-436c-947f-15f9753245f2 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:36.273105 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38730261-3655-436c-947f-15f9753245f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:36.277418 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:36.292697 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-38730261-3655-436c-947f-15f9753245f2 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:36.294079 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-38730261-3655-436c-947f-15f9753245f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:36.299751 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe662426-9eea-46c1-b3bd-73e18cae6c3d None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:36.341043 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840e864-86c2-4f9d-accf-e54f3c995e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:36.341609 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840e864-86c2-4f9d-accf-e54f3c995e12 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:36.342276 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840e864-86c2-4f9d-accf-e54f3c995e12 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:36.342276 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840e864-86c2-4f9d-accf-e54f3c995e12 None admin] PATH_INFO: `/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:36.343260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5840e864-86c2-4f9d-accf-e54f3c995e12 None admin] RBAC: Authorizing `identity:get_project(project_id=6ee4b811a8dc4bc39e6ed0d4275b1782)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:36.344808 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5840e864-86c2-4f9d-accf-e54f3c995e12 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:36.346589 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 173/353] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:30:36 2022] GET /identity/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:36.347195 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840e864-86c2-4f9d-accf-e54f3c995e12 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:36.384709 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 181/354] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:30:36.386218 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46ef82-b709-4ecb-a01e-9b4a37b37426 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:36.386507 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46ef82-b709-4ecb-a01e-9b4a37b37426 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:36.387634 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46ef82-b709-4ecb-a01e-9b4a37b37426 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:36.389064 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 182/355] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:36.389833 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab46ef82-b709-4ecb-a01e-9b4a37b37426 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:36.486518 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840e864-86c2-4f9d-accf-e54f3c995e12 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:36.500032 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:36.531166 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab46ef82-b709-4ecb-a01e-9b4a37b37426 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:36.549106 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c5e5e-c8bd-4e95-b589-f2549919f9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:36.550269 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c5e5e-c8bd-4e95-b589-f2549919f9e9 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:36.550724 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c5e5e-c8bd-4e95-b589-f2549919f9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:36.551451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c5e5e-c8bd-4e95-b589-f2549919f9e9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:36.565690 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c5e5e-c8bd-4e95-b589-f2549919f9e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:36.571193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-4f0c5e5e-c8bd-4e95-b589-f2549919f9e9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 12:30:36.571193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:36.571193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:36.571193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:36.571193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:36.571193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:36.571193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:36.571193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:36.571193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:36.571193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:36.571193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:36.571193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:36.571193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:36.571193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:36.571193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:36.571193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:36.572180 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:36.572180 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:36.572180 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:36.572180 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:36.572180 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:36.572180 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:36.572180 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:36.572180 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:36.572180 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:36.572180 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:36.572180 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:36.572180 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:36.572180 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:36.572180 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:36.572180 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:36.572180 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:36.573365 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:36.573365 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:36.573365 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:36.573365 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:36.573365 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:36.573365 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:36.573365 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:36.573365 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:36.573365 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:36.573365 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:36.573365 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 12:30:36.573365 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:30:36.573365 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 174/356] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:30:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:30:36.574001 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c5e5e-c8bd-4e95-b589-f2549919f9e9 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:36.576155 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6383026f-ef99-457b-b57c-35f64af1765f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:36.576388 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383026f-ef99-457b-b57c-35f64af1765f None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:36.576610 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383026f-ef99-457b-b57c-35f64af1765f None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:36.576884 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383026f-ef99-457b-b57c-35f64af1765f None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:36.577545 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6383026f-ef99-457b-b57c-35f64af1765f None admin] RBAC: Adding query filter params (name=member) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:36.577852 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6383026f-ef99-457b-b57c-35f64af1765f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:36.580509 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6383026f-ef99-457b-b57c-35f64af1765f None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:36.590105 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 183/357] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 12:30:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:36.591254 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6383026f-ef99-457b-b57c-35f64af1765f None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:36.595262 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:36.733227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6383026f-ef99-457b-b57c-35f64af1765f None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:36.735165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c5e5e-c8bd-4e95-b589-f2549919f9e9 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:36.779964 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a50c1f-3a6c-4de8-af20-54e20e66adf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:36.780733 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a50c1f-3a6c-4de8-af20-54e20e66adf0 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:36.781011 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a50c1f-3a6c-4de8-af20-54e20e66adf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:36.782801 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a50c1f-3a6c-4de8-af20-54e20e66adf0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:36.782932 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bbd701-9153-47db-8d2e-bf8ab33091c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:36.783024 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbd701-9153-47db-8d2e-bf8ab33091c5 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:36.783267 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbd701-9153-47db-8d2e-bf8ab33091c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:36.783594 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbd701-9153-47db-8d2e-bf8ab33091c5 None admin] PATH_INFO: `/v3/groups/93681966bd5a46368ddcce563e694a25` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:36.784523 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a50c1f-3a6c-4de8-af20-54e20e66adf0 None admin] RBAC: Adding query filter params (group.id=93681966bd5a46368ddcce563e694a25, role.id=47d5e2ea167241d5a27c8bbf4593eead, scope.project.id=6ee4b811a8dc4bc39e6ed0d4275b1782) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:36.784720 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bbd701-9153-47db-8d2e-bf8ab33091c5 None admin] RBAC: Authorizing `identity:get_group(group_id=93681966bd5a46368ddcce563e694a25)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:36.787380 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a50c1f-3a6c-4de8-af20-54e20e66adf0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:36.787543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bbd701-9153-47db-8d2e-bf8ab33091c5 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:36.787985 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a50c1f-3a6c-4de8-af20-54e20e66adf0 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:36.788542 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 184/358] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Oct 3 12:30:36 2022] GET /identity/v3/groups/93681966bd5a46368ddcce563e694a25 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:36.789741 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f806e2-6e7a-4cae-a990-2340ed549f7b None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:36.790277 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f806e2-6e7a-4cae-a990-2340ed549f7b None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:36.790724 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f806e2-6e7a-4cae-a990-2340ed549f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:36.797032 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 175/359] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Oct 3 12:30:36 2022] GET /identity/v3/role_assignments?group.id=93681966bd5a46368ddcce563e694a25&scope.project.id=6ee4b811a8dc4bc39e6ed0d4275b1782&role.id=47d5e2ea167241d5a27c8bbf4593eead => generated 260 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:36.799680 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a50c1f-3a6c-4de8-af20-54e20e66adf0 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:36.817644 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-c0f806e2-6e7a-4cae-a990-2340ed549f7b None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:36.818936 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-c0f806e2-6e7a-4cae-a990-2340ed549f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:36.831521 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:36.917495 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:36.931487 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 185/360] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:30:36.932443 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f806e2-6e7a-4cae-a990-2340ed549f7b None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:36.961162 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a50c1f-3a6c-4de8-af20-54e20e66adf0 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:37.012908 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cf714e-72a5-439a-b3eb-63aaf38a9990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:37.014016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf714e-72a5-439a-b3eb-63aaf38a9990 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:37.014489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf714e-72a5-439a-b3eb-63aaf38a9990 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:37.014952 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf714e-72a5-439a-b3eb-63aaf38a9990 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:37.015991 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cf714e-72a5-439a-b3eb-63aaf38a9990 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:37.016561 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cf714e-72a5-439a-b3eb-63aaf38a9990 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:37.018164 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cf714e-72a5-439a-b3eb-63aaf38a9990 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:37.034404 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 176/361] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 12:30:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:37.035790 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cf714e-72a5-439a-b3eb-63aaf38a9990 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:37.085619 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f806e2-6e7a-4cae-a990-2340ed549f7b None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:37.140942 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9136c-96f1-41b6-82b7-39ecec22edaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:37.141586 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9136c-96f1-41b6-82b7-39ecec22edaa None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:37.141961 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9136c-96f1-41b6-82b7-39ecec22edaa None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:37.142325 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9136c-96f1-41b6-82b7-39ecec22edaa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:37.171450 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:37.179804 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cf714e-72a5-439a-b3eb-63aaf38a9990 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:37.252728 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcedaaf-18c6-49b6-9185-0d3442e4dbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:37.253839 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcedaaf-18c6-49b6-9185-0d3442e4dbbc None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:37.254216 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcedaaf-18c6-49b6-9185-0d3442e4dbbc None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:37.254585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcedaaf-18c6-49b6-9185-0d3442e4dbbc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:37.262583 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:37.268859 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcedaaf-18c6-49b6-9185-0d3442e4dbbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:37.279453 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-4bcedaaf-18c6-49b6-9185-0d3442e4dbbc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 12:30:37.279453 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:37.279453 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:37.279453 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:37.279453 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:37.279453 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:37.279453 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:37.279453 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:37.279453 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:37.279453 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:37.279453 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:37.279453 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:37.279453 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:37.279453 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:37.279453 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:37.279453 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:37.280746 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:37.280746 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:37.280746 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:37.280746 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:37.280746 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:37.280746 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:37.280746 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:37.280746 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:37.280746 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:37.280746 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:37.280746 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:37.280746 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:37.280746 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:37.280746 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:37.280746 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:37.280746 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:37.283612 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:37.283612 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:37.283612 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:37.283612 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:37.283612 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:37.283612 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:37.283612 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:37.283612 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:37.283612 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:37.283612 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:37.283612 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 12:30:37.283612 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:30:37.283612 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 177/362] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 12:30:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:30:37.283612 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcedaaf-18c6-49b6-9185-0d3442e4dbbc None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:37.291277 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce9136c-96f1-41b6-82b7-39ecec22edaa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:37.292790 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce9136c-96f1-41b6-82b7-39ecec22edaa None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:37.430246 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcedaaf-18c6-49b6-9185-0d3442e4dbbc None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:37.438305 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 186/363] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:30:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 12:30:37.438472 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9136c-96f1-41b6-82b7-39ecec22edaa None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:37.478069 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c239652c-9a15-4650-8b33-3764e7ef8da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:37.478691 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239652c-9a15-4650-8b33-3764e7ef8da4 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:37.479067 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239652c-9a15-4650-8b33-3764e7ef8da4 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:37.479393 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239652c-9a15-4650-8b33-3764e7ef8da4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:37.480132 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c239652c-9a15-4650-8b33-3764e7ef8da4 None admin] RBAC: Adding query filter params (group.id=93681966bd5a46368ddcce563e694a25, role.id=73b935a1ebc448899bbfc2381168d1cd, scope.project.id=6ee4b811a8dc4bc39e6ed0d4275b1782) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:37.480451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c239652c-9a15-4650-8b33-3764e7ef8da4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:37.482015 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c239652c-9a15-4650-8b33-3764e7ef8da4 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:37.489060 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 178/364] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Oct 3 12:30:37 2022] GET /identity/v3/role_assignments?group.id=93681966bd5a46368ddcce563e694a25&scope.project.id=6ee4b811a8dc4bc39e6ed0d4275b1782&role.id=73b935a1ebc448899bbfc2381168d1cd => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:37.489957 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c239652c-9a15-4650-8b33-3764e7ef8da4 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:37.499607 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:37.599159 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:37.634040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9136c-96f1-41b6-82b7-39ecec22edaa None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:37.666246 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c239652c-9a15-4650-8b33-3764e7ef8da4 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:37.682754 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5407f5f0-030d-497a-a4cb-f064fc3b82cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:37.683484 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407f5f0-030d-497a-a4cb-f064fc3b82cc None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:37.683767 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407f5f0-030d-497a-a4cb-f064fc3b82cc None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:37.684039 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407f5f0-030d-497a-a4cb-f064fc3b82cc None admin] PATH_INFO: `/v3/users/d0692f0f27574133a16641b99a131804` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:37.687562 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5407f5f0-030d-497a-a4cb-f064fc3b82cc None admin] RBAC: Authorizing `identity:get_user(user_id=d0692f0f27574133a16641b99a131804)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:37.687562 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5407f5f0-030d-497a-a4cb-f064fc3b82cc None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:37.688571 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 187/365] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:30:37 2022] GET /identity/v3/users/d0692f0f27574133a16641b99a131804 => generated 293 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:37.691221 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5407f5f0-030d-497a-a4cb-f064fc3b82cc None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:37.740869 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0fa4aa-e217-412a-96a6-5d0ec2dc7abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:37.742162 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0fa4aa-e217-412a-96a6-5d0ec2dc7abe None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:37.742765 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0fa4aa-e217-412a-96a6-5d0ec2dc7abe None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:37.743261 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0fa4aa-e217-412a-96a6-5d0ec2dc7abe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:37.764472 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0fa4aa-e217-412a-96a6-5d0ec2dc7abe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:37.775627 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-1f0fa4aa-e217-412a-96a6-5d0ec2dc7abe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:30:37.775627 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:37.775627 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:37.775627 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:37.775627 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:37.775627 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:37.775627 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:37.775627 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:37.775627 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:37.775627 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:37.775627 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:37.775627 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:37.775627 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:37.775627 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:37.775627 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:37.775627 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:37.776574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:37.776574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:37.776574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:37.776574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:37.776574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:37.776574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:37.776574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:37.776574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:37.776574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:37.776574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:37.776574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:37.776574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:37.776574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:37.776574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:37.776574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:37.776574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:37.777950 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:37.777950 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:37.777950 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:37.777950 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:37.777950 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:37.777950 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:37.777950 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:37.777950 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:37.777950 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:37.777950 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:37.777950 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:30:37.777950 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:30:37.777950 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 179/366] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 12:30:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:30:37.778605 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0fa4aa-e217-412a-96a6-5d0ec2dc7abe None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:37.873522 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5407f5f0-030d-497a-a4cb-f064fc3b82cc None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:37.888095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:37.919539 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38faa287-d1d9-4b7c-83d4-07c98b9df157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:37.920279 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38faa287-d1d9-4b7c-83d4-07c98b9df157 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:37.920631 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38faa287-d1d9-4b7c-83d4-07c98b9df157 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:37.920978 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38faa287-d1d9-4b7c-83d4-07c98b9df157 None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:37.921744 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38faa287-d1d9-4b7c-83d4-07c98b9df157 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:37.922121 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38faa287-d1d9-4b7c-83d4-07c98b9df157 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:37.923615 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38faa287-d1d9-4b7c-83d4-07c98b9df157 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:37.937978 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 188/367] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:30:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:37.940036 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38faa287-d1d9-4b7c-83d4-07c98b9df157 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:37.947446 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0fa4aa-e217-412a-96a6-5d0ec2dc7abe None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:38.000929 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:38.018167 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d272120e-3971-416b-851b-001e0bbafb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:38.019834 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272120e-3971-416b-851b-001e0bbafb22 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:38.019834 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272120e-3971-416b-851b-001e0bbafb22 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:38.019834 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272120e-3971-416b-851b-001e0bbafb22 None admin] PATH_INFO: `/v3/projects/c974a924a68947af9e8abc10c2614ae7` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:38.022125 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d272120e-3971-416b-851b-001e0bbafb22 None admin] RBAC: Authorizing `identity:get_project(project_id=c974a924a68947af9e8abc10c2614ae7)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:38.024095 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d272120e-3971-416b-851b-001e0bbafb22 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:38.026870 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 180/368] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:30:37 2022] GET /identity/v3/projects/c974a924a68947af9e8abc10c2614ae7 => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:38.027823 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d272120e-3971-416b-851b-001e0bbafb22 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:38.086367 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38faa287-d1d9-4b7c-83d4-07c98b9df157 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:38.131929 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2464ca-c917-4ccf-a141-4f1524e9f38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:38.133219 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2464ca-c917-4ccf-a141-4f1524e9f38f None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:38.133650 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2464ca-c917-4ccf-a141-4f1524e9f38f None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:38.134068 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2464ca-c917-4ccf-a141-4f1524e9f38f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:38.134941 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2464ca-c917-4ccf-a141-4f1524e9f38f None admin] RBAC: Adding query filter params (role.id=5c094fe41dc347cbbb61cb9342eee721, user.id=d0692f0f27574133a16641b99a131804, scope.system=all) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:38.135959 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2464ca-c917-4ccf-a141-4f1524e9f38f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:38.137857 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2464ca-c917-4ccf-a141-4f1524e9f38f None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:38.150875 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d272120e-3971-416b-851b-001e0bbafb22 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:38.154747 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 189/369] 10.222.0.33 () {56 vars in 1308 bytes} [Mon Oct 3 12:30:37 2022] GET /identity/v3/role_assignments?user.id=d0692f0f27574133a16641b99a131804&scope.system=all&role.id=5c094fe41dc347cbbb61cb9342eee721 => generated 510 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:38.155860 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2464ca-c917-4ccf-a141-4f1524e9f38f None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:38.199906 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:38.268144 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d9b86-ac4d-4593-a83a-e7fc827e5cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:38.268803 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d9b86-ac4d-4593-a83a-e7fc827e5cec None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:38.269055 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d9b86-ac4d-4593-a83a-e7fc827e5cec None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:38.269324 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d9b86-ac4d-4593-a83a-e7fc827e5cec None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:38.269991 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17d9b86-ac4d-4593-a83a-e7fc827e5cec None admin] RBAC: Adding query filter params (name=admin) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:38.270291 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17d9b86-ac4d-4593-a83a-e7fc827e5cec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:38.271978 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17d9b86-ac4d-4593-a83a-e7fc827e5cec None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:38.285592 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2464ca-c917-4ccf-a141-4f1524e9f38f None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:38.293593 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 181/370] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:30:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:38.294374 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d9b86-ac4d-4593-a83a-e7fc827e5cec None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:38.334429 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c3bd95-6eb9-4c27-9e45-44700344aee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:38.335152 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3bd95-6eb9-4c27-9e45-44700344aee4 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:38.335462 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3bd95-6eb9-4c27-9e45-44700344aee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:38.335838 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3bd95-6eb9-4c27-9e45-44700344aee4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:38.398864 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:38.482407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c3bd95-6eb9-4c27-9e45-44700344aee4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:38.484885 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c3bd95-6eb9-4c27-9e45-44700344aee4 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:38.486338 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d9b86-ac4d-4593-a83a-e7fc827e5cec None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:38.538383 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:38.546501 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e430fe79-c98b-4318-a3e6-50179787abb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:38.548229 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430fe79-c98b-4318-a3e6-50179787abb7 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:38.548969 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430fe79-c98b-4318-a3e6-50179787abb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:38.549402 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430fe79-c98b-4318-a3e6-50179787abb7 None admin] PATH_INFO: `/v3/groups/93681966bd5a46368ddcce563e694a25` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:38.551828 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e430fe79-c98b-4318-a3e6-50179787abb7 None admin] RBAC: Authorizing `identity:get_group(group_id=93681966bd5a46368ddcce563e694a25)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:38.553998 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e430fe79-c98b-4318-a3e6-50179787abb7 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:38.555688 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 182/371] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Oct 3 12:30:38 2022] GET /identity/v3/groups/93681966bd5a46368ddcce563e694a25 => generated 226 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:38.556466 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e430fe79-c98b-4318-a3e6-50179787abb7 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:38.647144 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 190/372] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:30:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 12:30:38.648344 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c3bd95-6eb9-4c27-9e45-44700344aee4 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:38.691073 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e430fe79-c98b-4318-a3e6-50179787abb7 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:38.717347 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:38.735767 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519869ce-6528-4edf-82d9-2c4dd895b191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:38.736428 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519869ce-6528-4edf-82d9-2c4dd895b191 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:38.736816 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519869ce-6528-4edf-82d9-2c4dd895b191 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:38.737158 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519869ce-6528-4edf-82d9-2c4dd895b191 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:38.786757 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c3bd95-6eb9-4c27-9e45-44700344aee4 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:38.821221 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:38.825658 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519869ce-6528-4edf-82d9-2c4dd895b191 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:38.828736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c8804-8e65-473b-8349-fa3a70a279c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:38.829554 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c8804-8e65-473b-8349-fa3a70a279c2 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:38.829898 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519869ce-6528-4edf-82d9-2c4dd895b191 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:38.848040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c8804-8e65-473b-8349-fa3a70a279c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:38.848040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c8804-8e65-473b-8349-fa3a70a279c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:38.848040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87c8804-8e65-473b-8349-fa3a70a279c2 None admin] RBAC: Adding query filter params (group.id=93681966bd5a46368ddcce563e694a25, role.id=47d5e2ea167241d5a27c8bbf4593eead, scope.project.id=c974a924a68947af9e8abc10c2614ae7) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:38.848040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87c8804-8e65-473b-8349-fa3a70a279c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:38.848040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87c8804-8e65-473b-8349-fa3a70a279c2 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:38.848421 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 191/373] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Oct 3 12:30:38 2022] GET /identity/v3/role_assignments?group.id=93681966bd5a46368ddcce563e694a25&scope.project.id=c974a924a68947af9e8abc10c2614ae7&role.id=47d5e2ea167241d5a27c8bbf4593eead => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:38.848421 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c8804-8e65-473b-8349-fa3a70a279c2 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:38.969481 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c8804-8e65-473b-8349-fa3a70a279c2 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:39.012991 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f2f8b1-7ffd-4991-8688-15640869f97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:39.014464 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2f8b1-7ffd-4991-8688-15640869f97f None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:39.014944 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2f8b1-7ffd-4991-8688-15640869f97f None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:39.015432 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2f8b1-7ffd-4991-8688-15640869f97f None admin] PATH_INFO: `/v3/projects/c974a924a68947af9e8abc10c2614ae7` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:39.017052 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f2f8b1-7ffd-4991-8688-15640869f97f None admin] RBAC: Authorizing `identity:get_project(project_id=c974a924a68947af9e8abc10c2614ae7)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:39.018781 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f2f8b1-7ffd-4991-8688-15640869f97f None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:39.020776 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 192/374] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:30:38 2022] GET /identity/v3/projects/c974a924a68947af9e8abc10c2614ae7 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:39.028279 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f2f8b1-7ffd-4991-8688-15640869f97f None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:39.048299 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 183/375] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:30:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 12:30:39.054171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519869ce-6528-4edf-82d9-2c4dd895b191 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:39.110164 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:39.119928 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:39.173153 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f2f8b1-7ffd-4991-8688-15640869f97f None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:39.233655 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519869ce-6528-4edf-82d9-2c4dd895b191 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:39.257652 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00081a-d2a9-4db6-abc7-a65a4703dc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:39.258622 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00081a-d2a9-4db6-abc7-a65a4703dc8f None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:39.259086 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00081a-d2a9-4db6-abc7-a65a4703dc8f None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:39.259634 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00081a-d2a9-4db6-abc7-a65a4703dc8f None admin] PATH_INFO: `/v3/groups/93681966bd5a46368ddcce563e694a25` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:39.261129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e00081a-d2a9-4db6-abc7-a65a4703dc8f None admin] RBAC: Authorizing `identity:get_group(group_id=93681966bd5a46368ddcce563e694a25)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:39.265454 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e00081a-d2a9-4db6-abc7-a65a4703dc8f None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:39.267757 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 193/376] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Oct 3 12:30:39 2022] GET /identity/v3/groups/93681966bd5a46368ddcce563e694a25 => generated 226 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:39.275014 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00081a-d2a9-4db6-abc7-a65a4703dc8f None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:39.291000 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2cd39-a5f0-42d6-be1e-914742037932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:39.292068 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2cd39-a5f0-42d6-be1e-914742037932 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:39.292068 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2cd39-a5f0-42d6-be1e-914742037932 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:39.292068 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2cd39-a5f0-42d6-be1e-914742037932 None admin] PATH_INFO: `/v3/projects/912de75d08d24473bf98cc746c209242` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:39.296481 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c2cd39-a5f0-42d6-be1e-914742037932 None admin] RBAC: Authorizing `identity:get_project(project_id=912de75d08d24473bf98cc746c209242)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:39.296481 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c2cd39-a5f0-42d6-be1e-914742037932 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:39.296784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 184/377] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:30:39 2022] GET /identity/v3/projects/912de75d08d24473bf98cc746c209242 => generated 344 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:39.304238 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2cd39-a5f0-42d6-be1e-914742037932 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:39.433216 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:39.461749 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2cd39-a5f0-42d6-be1e-914742037932 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:39.497580 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00081a-d2a9-4db6-abc7-a65a4703dc8f None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:39.501066 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cbe4b8-2e37-4eb9-ae88-f2ab971369ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:39.501764 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cbe4b8-2e37-4eb9-ae88-f2ab971369ab None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:39.502039 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cbe4b8-2e37-4eb9-ae88-f2ab971369ab None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:39.502306 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cbe4b8-2e37-4eb9-ae88-f2ab971369ab None admin] PATH_INFO: `/v3/groups/26b4b111512144a28363e2bf8784330e` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:39.510299 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cbe4b8-2e37-4eb9-ae88-f2ab971369ab None admin] RBAC: Authorizing `identity:get_group(group_id=26b4b111512144a28363e2bf8784330e)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:39.512338 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cbe4b8-2e37-4eb9-ae88-f2ab971369ab None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:39.514456 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 185/378] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Oct 3 12:30:39 2022] GET /identity/v3/groups/26b4b111512144a28363e2bf8784330e => generated 229 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:39.522584 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cbe4b8-2e37-4eb9-ae88-f2ab971369ab None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:39.550464 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:39.562527 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627d2e96-c056-40a4-abfc-310167291321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:39.563154 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627d2e96-c056-40a4-abfc-310167291321 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:39.563497 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627d2e96-c056-40a4-abfc-310167291321 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:39.563775 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627d2e96-c056-40a4-abfc-310167291321 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:39.564511 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627d2e96-c056-40a4-abfc-310167291321 None admin] RBAC: Adding query filter params (group.id=93681966bd5a46368ddcce563e694a25, role.id=73b935a1ebc448899bbfc2381168d1cd, scope.project.id=c974a924a68947af9e8abc10c2614ae7) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:39.564877 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627d2e96-c056-40a4-abfc-310167291321 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:39.579427 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627d2e96-c056-40a4-abfc-310167291321 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:39.579427 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 194/379] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Oct 3 12:30:39 2022] GET /identity/v3/role_assignments?group.id=93681966bd5a46368ddcce563e694a25&scope.project.id=c974a924a68947af9e8abc10c2614ae7&role.id=73b935a1ebc448899bbfc2381168d1cd => generated 260 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:30:39.579427 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58fdd2-3c34-48ce-9c56-0c14e89afa7b None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:39.594805 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58fdd2-3c34-48ce-9c56-0c14e89afa7b None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:39.594805 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58fdd2-3c34-48ce-9c56-0c14e89afa7b None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:39.595194 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 195/380] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:39 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:39.603348 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd9cc9-9269-436d-b4ae-1db4c0c912fd None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:39.603348 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd9cc9-9269-436d-b4ae-1db4c0c912fd None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:39.603348 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd9cc9-9269-436d-b4ae-1db4c0c912fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:39.658715 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-5fbd9cc9-9269-436d-b4ae-1db4c0c912fd None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:39.663353 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-5fbd9cc9-9269-436d-b4ae-1db4c0c912fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:39.740868 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cbe4b8-2e37-4eb9-ae88-f2ab971369ab None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:39.823422 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 196/381] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:30:39.823608 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a900daae-daea-468a-a1a8-f25a76ccfd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:39.824046 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900daae-daea-468a-a1a8-f25a76ccfd7f None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:39.824331 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900daae-daea-468a-a1a8-f25a76ccfd7f None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:39.824603 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900daae-daea-468a-a1a8-f25a76ccfd7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:39.825343 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a900daae-daea-468a-a1a8-f25a76ccfd7f None admin] RBAC: Adding query filter params (group.id=26b4b111512144a28363e2bf8784330e, role.id=7933c45dcdd54ef0a98018ba8a8bddaa, scope.project.id=912de75d08d24473bf98cc746c209242) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:39.825651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a900daae-daea-468a-a1a8-f25a76ccfd7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:39.827107 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a900daae-daea-468a-a1a8-f25a76ccfd7f None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:39.847842 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 186/382] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Oct 3 12:30:39 2022] GET /identity/v3/role_assignments?group.id=26b4b111512144a28363e2bf8784330e&scope.project.id=912de75d08d24473bf98cc746c209242&role.id=7933c45dcdd54ef0a98018ba8a8bddaa => generated 260 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:40.186306 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5029cdea-e834-4e06-a8f2-1497318722d3 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:40.186809 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5029cdea-e834-4e06-a8f2-1497318722d3 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:40.186809 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5029cdea-e834-4e06-a8f2-1497318722d3 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:40.188117 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 197/383] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:40.197891 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83986730-002d-4fc4-b3be-04a9889d00e1 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:40.198165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83986730-002d-4fc4-b3be-04a9889d00e1 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:40.198439 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83986730-002d-4fc4-b3be-04a9889d00e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:40.228094 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-83986730-002d-4fc4-b3be-04a9889d00e1 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:40.229330 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-83986730-002d-4fc4-b3be-04a9889d00e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:40.257599 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:40.341697 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 187/384] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:30:40.359193 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5029cdea-e834-4e06-a8f2-1497318722d3 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:40.519920 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5029cdea-e834-4e06-a8f2-1497318722d3 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:40.555893 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:40.573757 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe4f3b-d43f-4a87-9e28-e0a5ba822112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:40.573982 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe4f3b-d43f-4a87-9e28-e0a5ba822112 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:40.574926 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe4f3b-d43f-4a87-9e28-e0a5ba822112 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:40.574926 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe4f3b-d43f-4a87-9e28-e0a5ba822112 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:40.618765 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe4f3b-d43f-4a87-9e28-e0a5ba822112 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:40.623736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-8bfe4f3b-d43f-4a87-9e28-e0a5ba822112 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 12:30:40.623736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:40.623736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:40.623736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:40.623736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:40.623736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:40.623736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:40.623736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:40.623736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:40.623736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:40.623736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:40.623736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:40.623736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:40.623736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:40.623736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:40.623736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:40.624660 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:40.624660 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:40.624660 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:40.624660 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:40.624660 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:40.624660 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:40.624660 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:40.624660 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:40.624660 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:40.624660 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:40.624660 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:40.624660 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:40.624660 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:40.624660 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:40.624660 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:40.624660 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:40.625503 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:40.625503 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:40.625503 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:40.625503 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:40.625503 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:40.625503 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:40.625503 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:40.625503 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:40.625503 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:40.625503 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:40.625503 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 12:30:40.625503 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:30:40.628015 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 198/385] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:30:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:30:40.637391 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83986730-002d-4fc4-b3be-04a9889d00e1 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:40.719814 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe4f3b-d43f-4a87-9e28-e0a5ba822112 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:40.859760 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83986730-002d-4fc4-b3be-04a9889d00e1 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:40.940024 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe4f3b-d43f-4a87-9e28-e0a5ba822112 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:40.948502 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78facb8e-c4f1-475a-a96a-983733e3970e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:40.951338 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78facb8e-c4f1-475a-a96a-983733e3970e None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:40.951338 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78facb8e-c4f1-475a-a96a-983733e3970e None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:40.951338 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78facb8e-c4f1-475a-a96a-983733e3970e None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:40.951338 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78facb8e-c4f1-475a-a96a-983733e3970e None admin] RBAC: Adding query filter params (name=member) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:40.951338 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78facb8e-c4f1-475a-a96a-983733e3970e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:40.954837 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78facb8e-c4f1-475a-a96a-983733e3970e None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:40.968793 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 188/386] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 12:30:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:40.978045 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78facb8e-c4f1-475a-a96a-983733e3970e None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:40.999385 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:41.030500 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e80d7-e1c3-48cc-b90b-ee91a4942efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:41.032158 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e80d7-e1c3-48cc-b90b-ee91a4942efd None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:41.032158 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e80d7-e1c3-48cc-b90b-ee91a4942efd None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:41.032158 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e80d7-e1c3-48cc-b90b-ee91a4942efd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:41.058480 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132e80d7-e1c3-48cc-b90b-ee91a4942efd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:41.071422 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-132e80d7-e1c3-48cc-b90b-ee91a4942efd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 12:30:41.071422 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:41.071422 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:41.071422 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:41.071422 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:41.071422 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:41.071422 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:41.071422 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:41.071422 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:41.071422 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:41.071422 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:41.071422 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:41.071422 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:41.071422 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:41.071422 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:41.071422 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:41.078361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:41.078361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:41.078361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:41.078361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:41.078361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:41.078361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:41.078361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:41.078361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:41.078361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:41.078361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:41.078361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:41.078361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:41.078361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:41.078361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:41.078361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:41.078361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:41.079171 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:41.079171 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:41.079171 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:41.079171 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:41.079171 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:41.079171 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:41.079171 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:41.079171 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:41.079171 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:41.079171 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:41.079171 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 12:30:41.079171 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:30:41.079171 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 199/387] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 12:30:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:30:41.092431 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e80d7-e1c3-48cc-b90b-ee91a4942efd None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:41.172859 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78facb8e-c4f1-475a-a96a-983733e3970e None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:41.243003 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d08d9-2b00-478d-9126-212fff3d3a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:41.243003 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d08d9-2b00-478d-9126-212fff3d3a43 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:41.243003 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d08d9-2b00-478d-9126-212fff3d3a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:41.243003 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d08d9-2b00-478d-9126-212fff3d3a43 None admin] PATH_INFO: `/v3/groups/93681966bd5a46368ddcce563e694a25` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:41.244054 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8d08d9-2b00-478d-9126-212fff3d3a43 None admin] RBAC: Authorizing `identity:get_group(group_id=93681966bd5a46368ddcce563e694a25)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:41.245072 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8d08d9-2b00-478d-9126-212fff3d3a43 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:41.249739 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 189/388] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Oct 3 12:30:40 2022] GET /identity/v3/groups/93681966bd5a46368ddcce563e694a25 => generated 226 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:41.259967 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:41.260551 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d08d9-2b00-478d-9126-212fff3d3a43 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:41.364253 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e80d7-e1c3-48cc-b90b-ee91a4942efd None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:41.439701 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:41.452118 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe554ea3-f13a-4d4d-b044-29f9fafab65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:41.452752 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe554ea3-f13a-4d4d-b044-29f9fafab65f None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:41.453033 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe554ea3-f13a-4d4d-b044-29f9fafab65f None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:41.453307 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe554ea3-f13a-4d4d-b044-29f9fafab65f None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:41.453980 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe554ea3-f13a-4d4d-b044-29f9fafab65f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:41.454291 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe554ea3-f13a-4d4d-b044-29f9fafab65f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:41.461124 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe554ea3-f13a-4d4d-b044-29f9fafab65f None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:41.481730 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 200/389] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:30:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:41.484763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe554ea3-f13a-4d4d-b044-29f9fafab65f None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:41.488617 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d08d9-2b00-478d-9126-212fff3d3a43 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:41.565583 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b037dc08-d588-4b39-a328-4766dd48fd1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:41.566280 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037dc08-d588-4b39-a328-4766dd48fd1a None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:41.566564 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037dc08-d588-4b39-a328-4766dd48fd1a None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:41.566841 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037dc08-d588-4b39-a328-4766dd48fd1a None admin] PATH_INFO: `/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:41.568062 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b037dc08-d588-4b39-a328-4766dd48fd1a None admin] RBAC: Authorizing `identity:get_project(project_id=6ee4b811a8dc4bc39e6ed0d4275b1782)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:41.571476 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b037dc08-d588-4b39-a328-4766dd48fd1a None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:41.573468 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 190/390] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:30:41 2022] GET /identity/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782 => generated 298 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:30:41.580072 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b037dc08-d588-4b39-a328-4766dd48fd1a None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:41.717775 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe554ea3-f13a-4d4d-b044-29f9fafab65f None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:41.738179 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:41.770539 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b037dc08-d588-4b39-a328-4766dd48fd1a None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:41.799151 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d4686-62ca-403b-8dba-d77e0721372b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:41.800684 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d4686-62ca-403b-8dba-d77e0721372b None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:41.801468 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d4686-62ca-403b-8dba-d77e0721372b None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:41.803840 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d4686-62ca-403b-8dba-d77e0721372b None admin] PATH_INFO: `/v3/groups/93681966bd5a46368ddcce563e694a25` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:41.805156 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d4686-62ca-403b-8dba-d77e0721372b None admin] RBAC: Authorizing `identity:get_group(group_id=93681966bd5a46368ddcce563e694a25)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:41.807856 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d4686-62ca-403b-8dba-d77e0721372b None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:41.810005 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 201/391] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Oct 3 12:30:41 2022] GET /identity/v3/groups/93681966bd5a46368ddcce563e694a25 => generated 226 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:41.811916 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f03bab3-416d-4c08-801d-77f20974a614 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:41.812194 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f03bab3-416d-4c08-801d-77f20974a614 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:41.812465 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f03bab3-416d-4c08-801d-77f20974a614 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:41.819113 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 202/392] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:41.820835 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f03bab3-416d-4c08-801d-77f20974a614 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:41.823341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1054c1-e117-4149-97f4-596c45db2edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:41.824024 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1054c1-e117-4149-97f4-596c45db2edf None admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:41.824300 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1054c1-e117-4149-97f4-596c45db2edf None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:41.824570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1054c1-e117-4149-97f4-596c45db2edf None admin] PATH_INFO: `/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782/groups/93681966bd5a46368ddcce563e694a25/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:41.828210 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1054c1-e117-4149-97f4-596c45db2edf None admin] RBAC: Authorizing `identity:create_grant(project_id=6ee4b811a8dc4bc39e6ed0d4275b1782, group_id=93681966bd5a46368ddcce563e694a25, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:41.829691 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1054c1-e117-4149-97f4-596c45db2edf None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:41.910481 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 191/393] 10.222.0.33 () {58 vars in 1442 bytes} [Mon Oct 3 12:30:41 2022] PUT /identity/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782/groups/93681966bd5a46368ddcce563e694a25/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:30:41.911897 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd004a-6789-492c-87d3-63e8d2d3d105 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:41.912320 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd004a-6789-492c-87d3-63e8d2d3d105 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:41.914025 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd004a-6789-492c-87d3-63e8d2d3d105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:41.952747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:41.967094 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f03bab3-416d-4c08-801d-77f20974a614 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:41.980575 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-9bdd004a-6789-492c-87d3-63e8d2d3d105 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:41.981732 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-9bdd004a-6789-492c-87d3-63e8d2d3d105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:42.048793 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f9f439-b025-4f03-a33c-eaef81e1f512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:42.049618 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9f439-b025-4f03-a33c-eaef81e1f512 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:42.049735 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9f439-b025-4f03-a33c-eaef81e1f512 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:42.049975 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9f439-b025-4f03-a33c-eaef81e1f512 None admin] PATH_INFO: `/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:42.052049 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f9f439-b025-4f03-a33c-eaef81e1f512 None admin] RBAC: Authorizing `identity:get_project(project_id=6ee4b811a8dc4bc39e6ed0d4275b1782)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:42.058240 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f9f439-b025-4f03-a33c-eaef81e1f512 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:42.060262 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 203/394] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:30:41 2022] GET /identity/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782 => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:42.066037 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f9f439-b025-4f03-a33c-eaef81e1f512 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:42.146003 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 192/395] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:30:42.160160 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:42.235266 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f9f439-b025-4f03-a33c-eaef81e1f512 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:42.291405 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebf2d65-4620-4f25-aeae-9776e587410a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:42.291405 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf2d65-4620-4f25-aeae-9776e587410a None admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:42.291405 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf2d65-4620-4f25-aeae-9776e587410a None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:42.291405 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf2d65-4620-4f25-aeae-9776e587410a None admin] PATH_INFO: `/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782/groups/93681966bd5a46368ddcce563e694a25/roles/73b935a1ebc448899bbfc2381168d1cd` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:42.293674 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebf2d65-4620-4f25-aeae-9776e587410a None admin] RBAC: Authorizing `identity:create_grant(project_id=6ee4b811a8dc4bc39e6ed0d4275b1782, group_id=93681966bd5a46368ddcce563e694a25, role_id=73b935a1ebc448899bbfc2381168d1cd)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:42.296727 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebf2d65-4620-4f25-aeae-9776e587410a None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:42.383038 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 204/396] 10.222.0.33 () {58 vars in 1442 bytes} [Mon Oct 3 12:30:42 2022] PUT /identity/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782/groups/93681966bd5a46368ddcce563e694a25/roles/73b935a1ebc448899bbfc2381168d1cd => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:30:42.477656 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d822a54-640b-47aa-884d-c5f1e0c2f7b0 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:42.478183 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d822a54-640b-47aa-884d-c5f1e0c2f7b0 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:42.478183 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d822a54-640b-47aa-884d-c5f1e0c2f7b0 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:42.483436 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 193/397] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:42.495879 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2eefb5-1d9f-4954-bdda-31d09075eb69 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:42.496061 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2eefb5-1d9f-4954-bdda-31d09075eb69 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:42.499033 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2eefb5-1d9f-4954-bdda-31d09075eb69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:42.511147 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d822a54-640b-47aa-884d-c5f1e0c2f7b0 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:42.540523 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-eb2eefb5-1d9f-4954-bdda-31d09075eb69 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:42.543171 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-eb2eefb5-1d9f-4954-bdda-31d09075eb69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:42.670770 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 205/398] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:30:42.788951 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874f676-46f5-4b6a-ab4e-febbf9b69698 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:42.789613 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874f676-46f5-4b6a-ab4e-febbf9b69698 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:42.790102 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874f676-46f5-4b6a-ab4e-febbf9b69698 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:42.793230 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 206/399] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:42.798280 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2a877-6ec5-4433-ab7d-dc3a7591342d None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:42.798807 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2a877-6ec5-4433-ab7d-dc3a7591342d None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:42.799290 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2a877-6ec5-4433-ab7d-dc3a7591342d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:42.816936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d822a54-640b-47aa-884d-c5f1e0c2f7b0 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:42.836756 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-05f2a877-6ec5-4433-ab7d-dc3a7591342d None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:42.839748 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-05f2a877-6ec5-4433-ab7d-dc3a7591342d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:42.889520 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f7336d-6180-46b7-b366-3c2777379067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:42.891719 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7336d-6180-46b7-b366-3c2777379067 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:42.892324 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7336d-6180-46b7-b366-3c2777379067 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:42.893025 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7336d-6180-46b7-b366-3c2777379067 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:42.899666 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:42.904464 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:42.918371 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f7336d-6180-46b7-b366-3c2777379067 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:42.929766 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-44f7336d-6180-46b7-b366-3c2777379067 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 12:30:42.929766 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:42.929766 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:42.929766 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:42.929766 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:42.929766 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:42.929766 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:42.929766 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:42.929766 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:42.929766 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:42.929766 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:42.929766 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:42.929766 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:42.929766 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:42.929766 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:42.929766 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:42.930759 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:42.930759 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:42.930759 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:42.930759 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:42.930759 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:42.930759 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:42.930759 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:42.930759 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:42.930759 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:42.930759 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:42.930759 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:42.930759 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:42.930759 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:42.930759 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:42.930759 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:42.930759 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:42.931574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:42.931574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:42.931574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:42.931574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:42.931574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:42.931574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:42.931574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:42.931574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:42.931574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:42.931574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:42.931574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 12:30:42.931574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:30:42.935353 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 194/400] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:30:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 423 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:30:42.945472 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f7336d-6180-46b7-b366-3c2777379067 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:42.981063 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 207/401] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 12:30:43.042253 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f2a877-6ec5-4433-ab7d-dc3a7591342d None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:43.160903 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f7336d-6180-46b7-b366-3c2777379067 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:43.202526 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61797500-8530-4931-afe8-7436ff11c477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:43.203146 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61797500-8530-4931-afe8-7436ff11c477 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:43.203462 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61797500-8530-4931-afe8-7436ff11c477 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:43.203867 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61797500-8530-4931-afe8-7436ff11c477 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:43.204535 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61797500-8530-4931-afe8-7436ff11c477 None admin] RBAC: Adding query filter params (name=member) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:43.205196 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61797500-8530-4931-afe8-7436ff11c477 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:43.206553 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61797500-8530-4931-afe8-7436ff11c477 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:43.216967 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 195/402] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 12:30:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:30:43.222998 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61797500-8530-4931-afe8-7436ff11c477 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:43.294510 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:43.338379 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f2a877-6ec5-4433-ab7d-dc3a7591342d None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:43.356567 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61797500-8530-4931-afe8-7436ff11c477 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:43.394884 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19458e39-4ed7-49bf-bcf7-564bc8060d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:43.398674 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19458e39-4ed7-49bf-bcf7-564bc8060d58 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:43.398965 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19458e39-4ed7-49bf-bcf7-564bc8060d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:43.399238 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19458e39-4ed7-49bf-bcf7-564bc8060d58 None admin] PATH_INFO: `/v3/groups/93681966bd5a46368ddcce563e694a25` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:43.400969 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19458e39-4ed7-49bf-bcf7-564bc8060d58 None admin] RBAC: Authorizing `identity:get_group(group_id=93681966bd5a46368ddcce563e694a25)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:43.402415 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19458e39-4ed7-49bf-bcf7-564bc8060d58 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:43.404007 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ea18c-ae4a-4e21-b9e4-21f91f0427e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:43.404655 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ea18c-ae4a-4e21-b9e4-21f91f0427e0 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:43.404932 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ea18c-ae4a-4e21-b9e4-21f91f0427e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:43.405201 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ea18c-ae4a-4e21-b9e4-21f91f0427e0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:43.411029 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 196/403] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Oct 3 12:30:43 2022] GET /identity/v3/groups/93681966bd5a46368ddcce563e694a25 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:43.412125 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19458e39-4ed7-49bf-bcf7-564bc8060d58 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:43.426802 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ea18c-ae4a-4e21-b9e4-21f91f0427e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:43.431769 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-8e9ea18c-ae4a-4e21-b9e4-21f91f0427e0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 12:30:43.431769 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:43.431769 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:43.431769 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:43.431769 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:43.431769 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:43.431769 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:43.431769 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:43.431769 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:43.431769 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:43.431769 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:43.431769 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:43.431769 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:43.431769 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:43.431769 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:43.431769 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:43.432628 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:43.432628 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:43.432628 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:43.432628 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:43.432628 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:43.432628 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:43.432628 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:43.432628 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:43.432628 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:43.432628 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:43.432628 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:43.432628 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:43.432628 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:43.432628 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:43.432628 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:43.432628 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:43.433405 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:43.433405 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:43.433405 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:43.433405 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:43.433405 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:43.433405 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:43.433405 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:43.433405 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:43.433405 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:43.433405 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:43.433405 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 12:30:43.433405 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:30:43.433405 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 208/404] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 12:30:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:30:43.434051 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ea18c-ae4a-4e21-b9e4-21f91f0427e0 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:43.486747 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:43.562066 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19458e39-4ed7-49bf-bcf7-564bc8060d58 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:43.591504 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:43.604800 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff99dc7-3388-46b4-8f71-f163fa081f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:43.606300 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff99dc7-3388-46b4-8f71-f163fa081f9a None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:43.606777 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff99dc7-3388-46b4-8f71-f163fa081f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:43.607254 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff99dc7-3388-46b4-8f71-f163fa081f9a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:43.619281 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ea18c-ae4a-4e21-b9e4-21f91f0427e0 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:43.619768 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff99dc7-3388-46b4-8f71-f163fa081f9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:43.626833 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-cff99dc7-3388-46b4-8f71-f163fa081f9a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:30:43.626833 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:43.626833 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:43.626833 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:43.626833 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:43.626833 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:43.626833 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:43.626833 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:43.626833 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:43.626833 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:43.626833 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:43.626833 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:43.626833 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:43.626833 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:43.626833 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:43.626833 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:43.627757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:43.627757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:43.627757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:43.627757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:43.627757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:43.627757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:43.627757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:43.627757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:43.627757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:43.627757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:43.627757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:43.627757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:43.627757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:43.627757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:43.627757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:43.627757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:43.628520 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:43.628520 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:43.628520 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:43.628520 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:43.628520 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:43.628520 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:43.628520 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:43.628520 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:43.628520 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:43.628520 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:43.628520 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:30:43.628520 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:30:43.631671 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 197/405] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 12:30:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:30:43.632657 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff99dc7-3388-46b4-8f71-f163fa081f9a None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:43.692197 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1791dfb8-4dc4-4d27-9c07-b32eba181837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:43.692649 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1791dfb8-4dc4-4d27-9c07-b32eba181837 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:43.693467 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1791dfb8-4dc4-4d27-9c07-b32eba181837 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:43.693754 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1791dfb8-4dc4-4d27-9c07-b32eba181837 None admin] PATH_INFO: `/v3/projects/c974a924a68947af9e8abc10c2614ae7` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:43.695678 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1791dfb8-4dc4-4d27-9c07-b32eba181837 None admin] RBAC: Authorizing `identity:get_project(project_id=c974a924a68947af9e8abc10c2614ae7)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:43.700543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1791dfb8-4dc4-4d27-9c07-b32eba181837 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:43.710724 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 209/406] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:30:43 2022] GET /identity/v3/projects/c974a924a68947af9e8abc10c2614ae7 => generated 302 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:43.710724 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1791dfb8-4dc4-4d27-9c07-b32eba181837 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:43.754317 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff99dc7-3388-46b4-8f71-f163fa081f9a None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:43.798641 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ebf42a-89d2-457c-9743-e1c73a451cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:43.799648 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ebf42a-89d2-457c-9743-e1c73a451cad None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:43.800120 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ebf42a-89d2-457c-9743-e1c73a451cad None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:43.803883 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ebf42a-89d2-457c-9743-e1c73a451cad None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:43.804771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ebf42a-89d2-457c-9743-e1c73a451cad None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:43.805270 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ebf42a-89d2-457c-9743-e1c73a451cad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:43.806936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ebf42a-89d2-457c-9743-e1c73a451cad None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:43.821034 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 198/407] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:30:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:43.822157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ebf42a-89d2-457c-9743-e1c73a451cad None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:43.878412 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:43.880053 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:43.898359 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1791dfb8-4dc4-4d27-9c07-b32eba181837 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:43.952939 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ebf42a-89d2-457c-9743-e1c73a451cad None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:43.980215 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8affe7c-26a5-48f3-9095-71e6c7992cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:43.980973 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8affe7c-26a5-48f3-9095-71e6c7992cab None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:43.981152 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8affe7c-26a5-48f3-9095-71e6c7992cab None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:43.981403 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8affe7c-26a5-48f3-9095-71e6c7992cab None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:43.982085 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8affe7c-26a5-48f3-9095-71e6c7992cab None admin] RBAC: Adding query filter params (name=admin) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:43.982460 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8affe7c-26a5-48f3-9095-71e6c7992cab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:43.983799 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8affe7c-26a5-48f3-9095-71e6c7992cab None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:44.002394 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f9ef7-8e88-45bc-bda1-5699ddb87590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:44.003524 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f9ef7-8e88-45bc-bda1-5699ddb87590 None admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:44.004066 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f9ef7-8e88-45bc-bda1-5699ddb87590 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:44.004532 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f9ef7-8e88-45bc-bda1-5699ddb87590 None admin] PATH_INFO: `/v3/projects/c974a924a68947af9e8abc10c2614ae7/groups/93681966bd5a46368ddcce563e694a25/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:44.010695 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907f9ef7-8e88-45bc-bda1-5699ddb87590 None admin] RBAC: Authorizing `identity:create_grant(project_id=c974a924a68947af9e8abc10c2614ae7, group_id=93681966bd5a46368ddcce563e694a25, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:44.012521 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907f9ef7-8e88-45bc-bda1-5699ddb87590 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:44.012735 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 210/408] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:30:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:44.013766 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8affe7c-26a5-48f3-9095-71e6c7992cab None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:44.097888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 199/409] 10.222.0.33 () {58 vars in 1442 bytes} [Mon Oct 3 12:30:43 2022] PUT /identity/v3/projects/c974a924a68947af9e8abc10c2614ae7/groups/93681966bd5a46368ddcce563e694a25/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:30:44.104681 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f9ef7-8e88-45bc-bda1-5699ddb87590 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:44.212391 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8affe7c-26a5-48f3-9095-71e6c7992cab None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:44.312755 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f9ef7-8e88-45bc-bda1-5699ddb87590 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:44.320276 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4754b1-7201-4dc2-8023-cc6823f185e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:44.321054 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4754b1-7201-4dc2-8023-cc6823f185e0 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:44.321410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4754b1-7201-4dc2-8023-cc6823f185e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:44.321759 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4754b1-7201-4dc2-8023-cc6823f185e0 None admin] PATH_INFO: `/v3/groups/93681966bd5a46368ddcce563e694a25` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:44.327354 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:44.327569 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4754b1-7201-4dc2-8023-cc6823f185e0 None admin] RBAC: Authorizing `identity:get_group(group_id=93681966bd5a46368ddcce563e694a25)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:44.327569 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4754b1-7201-4dc2-8023-cc6823f185e0 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:44.336469 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 211/410] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Oct 3 12:30:44 2022] GET /identity/v3/groups/93681966bd5a46368ddcce563e694a25 => generated 226 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:44.347179 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4754b1-7201-4dc2-8023-cc6823f185e0 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:44.371935 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e6812-7eb5-475d-ba6d-7603b825cb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:44.372579 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e6812-7eb5-475d-ba6d-7603b825cb8b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:44.373323 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e6812-7eb5-475d-ba6d-7603b825cb8b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:44.373323 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e6812-7eb5-475d-ba6d-7603b825cb8b None admin] PATH_INFO: `/v3/groups/26b4b111512144a28363e2bf8784330e` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:44.376148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e6812-7eb5-475d-ba6d-7603b825cb8b None admin] RBAC: Authorizing `identity:get_group(group_id=26b4b111512144a28363e2bf8784330e)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:44.378869 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e6812-7eb5-475d-ba6d-7603b825cb8b None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:44.383940 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 200/411] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Oct 3 12:30:44 2022] GET /identity/v3/groups/26b4b111512144a28363e2bf8784330e => generated 229 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:44.391141 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e6812-7eb5-475d-ba6d-7603b825cb8b None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:44.417126 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:44.586620 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4754b1-7201-4dc2-8023-cc6823f185e0 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:44.616166 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e6812-7eb5-475d-ba6d-7603b825cb8b None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:44.668704 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28419dc3-e10f-4f63-a6fb-40b86f4c0ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:44.669383 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28419dc3-e10f-4f63-a6fb-40b86f4c0ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:44.669737 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28419dc3-e10f-4f63-a6fb-40b86f4c0ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:44.670013 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28419dc3-e10f-4f63-a6fb-40b86f4c0ea9 None admin] PATH_INFO: `/v3/projects/c974a924a68947af9e8abc10c2614ae7` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:44.671170 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28419dc3-e10f-4f63-a6fb-40b86f4c0ea9 None admin] RBAC: Authorizing `identity:get_project(project_id=c974a924a68947af9e8abc10c2614ae7)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:44.672652 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28419dc3-e10f-4f63-a6fb-40b86f4c0ea9 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:44.680507 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 212/412] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:30:44 2022] GET /identity/v3/projects/c974a924a68947af9e8abc10c2614ae7 => generated 302 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:44.682944 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28419dc3-e10f-4f63-a6fb-40b86f4c0ea9 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:44.694986 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa048e-e454-4f1d-ad04-272fe63c21e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:44.695706 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa048e-e454-4f1d-ad04-272fe63c21e9 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:44.695991 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa048e-e454-4f1d-ad04-272fe63c21e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:44.696378 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa048e-e454-4f1d-ad04-272fe63c21e9 None admin] PATH_INFO: `/v3/projects/912de75d08d24473bf98cc746c209242` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:44.704483 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fa048e-e454-4f1d-ad04-272fe63c21e9 None admin] RBAC: Authorizing `identity:get_project(project_id=912de75d08d24473bf98cc746c209242)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:44.705966 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fa048e-e454-4f1d-ad04-272fe63c21e9 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:44.708063 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 201/413] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:30:44 2022] GET /identity/v3/projects/912de75d08d24473bf98cc746c209242 => generated 344 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:30:44.716741 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa048e-e454-4f1d-ad04-272fe63c21e9 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:44.800370 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:44.858082 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:44.877109 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28419dc3-e10f-4f63-a6fb-40b86f4c0ea9 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:44.924750 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa048e-e454-4f1d-ad04-272fe63c21e9 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:44.926385 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e7cc8b-19f8-4580-948d-41a6dc6f9b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:44.927513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7cc8b-19f8-4580-948d-41a6dc6f9b81 None admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:44.928754 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7cc8b-19f8-4580-948d-41a6dc6f9b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:44.929242 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7cc8b-19f8-4580-948d-41a6dc6f9b81 None admin] PATH_INFO: `/v3/projects/c974a924a68947af9e8abc10c2614ae7/groups/93681966bd5a46368ddcce563e694a25/roles/73b935a1ebc448899bbfc2381168d1cd` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:44.932649 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e7cc8b-19f8-4580-948d-41a6dc6f9b81 None admin] RBAC: Authorizing `identity:create_grant(project_id=c974a924a68947af9e8abc10c2614ae7, group_id=93681966bd5a46368ddcce563e694a25, role_id=73b935a1ebc448899bbfc2381168d1cd)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:44.934113 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e7cc8b-19f8-4580-948d-41a6dc6f9b81 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:44.972902 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ec334-8a95-422f-8698-a835b8743fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:44.973569 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ec334-8a95-422f-8698-a835b8743fab None admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:44.974082 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ec334-8a95-422f-8698-a835b8743fab None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:44.974952 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ec334-8a95-422f-8698-a835b8743fab None admin] PATH_INFO: `/v3/projects/912de75d08d24473bf98cc746c209242/groups/26b4b111512144a28363e2bf8784330e/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:44.977033 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ec334-8a95-422f-8698-a835b8743fab None admin] RBAC: Authorizing `identity:create_grant(project_id=912de75d08d24473bf98cc746c209242, group_id=26b4b111512144a28363e2bf8784330e, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:44.978695 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ec334-8a95-422f-8698-a835b8743fab None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:45.009659 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 213/414] 10.222.0.33 () {58 vars in 1442 bytes} [Mon Oct 3 12:30:44 2022] PUT /identity/v3/projects/c974a924a68947af9e8abc10c2614ae7/groups/93681966bd5a46368ddcce563e694a25/roles/73b935a1ebc448899bbfc2381168d1cd => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:30:45.010901 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c17d0d-2b52-403b-b02a-0fb76ce9d5fb None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:45.011187 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c17d0d-2b52-403b-b02a-0fb76ce9d5fb None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:45.011546 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c17d0d-2b52-403b-b02a-0fb76ce9d5fb None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:45.016962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 214/415] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:45.016962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac3ce3-671e-4191-b4a4-07f9f4984c93 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:45.016962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac3ce3-671e-4191-b4a4-07f9f4984c93 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:45.016962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac3ce3-671e-4191-b4a4-07f9f4984c93 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:45.020587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 215/416] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:45.020998 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0a7aa-fe6c-4519-b6c5-d5bb7dff632d None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:45.021952 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0a7aa-fe6c-4519-b6c5-d5bb7dff632d None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:45.022253 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0a7aa-fe6c-4519-b6c5-d5bb7dff632d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:45.043285 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-daf0a7aa-fe6c-4519-b6c5-d5bb7dff632d None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:45.044726 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-daf0a7aa-fe6c-4519-b6c5-d5bb7dff632d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:45.085800 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 202/417] 10.222.0.33 () {58 vars in 1442 bytes} [Mon Oct 3 12:30:44 2022] PUT /identity/v3/projects/912de75d08d24473bf98cc746c209242/groups/26b4b111512144a28363e2bf8784330e/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:30:45.090208 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f75fe-d4e1-4530-982b-e363e8a118ca None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:45.090496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f75fe-d4e1-4530-982b-e363e8a118ca None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:45.090775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f75fe-d4e1-4530-982b-e363e8a118ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:45.133475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-ce5f75fe-d4e1-4530-982b-e363e8a118ca None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:45.137916 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ce5f75fe-d4e1-4530-982b-e363e8a118ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:45.177824 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 216/418] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:30:45.292478 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 203/419] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:30:45.522913 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf0a7aa-fe6c-4519-b6c5-d5bb7dff632d None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:45.633264 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f75fe-d4e1-4530-982b-e363e8a118ca None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:45.682933 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:45.699345 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:45.735220 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf0a7aa-fe6c-4519-b6c5-d5bb7dff632d None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:45.794333 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03daa3a6-5d76-4dff-9f77-cd3cc5e2955e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:45.795293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03daa3a6-5d76-4dff-9f77-cd3cc5e2955e None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:45.795816 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03daa3a6-5d76-4dff-9f77-cd3cc5e2955e None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:45.796291 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03daa3a6-5d76-4dff-9f77-cd3cc5e2955e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:45.814425 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03daa3a6-5d76-4dff-9f77-cd3cc5e2955e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:45.822108 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-03daa3a6-5d76-4dff-9f77-cd3cc5e2955e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 12:30:45.822108 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:45.822108 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:45.822108 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:45.822108 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:45.822108 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:45.822108 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:45.822108 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:45.822108 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:45.822108 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:45.822108 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:45.822108 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:45.822108 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:45.822108 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:45.822108 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:45.822108 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:45.823003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:45.823003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:45.823003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:45.823003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:45.823003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:45.823003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:45.823003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:45.823003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:45.823003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:45.823003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:45.823003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:45.823003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:45.823003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:45.823003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:45.823003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:45.823003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:45.823823 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:45.823823 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:45.823823 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:45.823823 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:45.823823 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:45.823823 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:45.823823 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:45.823823 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:45.823823 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:45.823823 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:45.823823 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 12:30:45.823823 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:30:45.826608 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 217/420] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:30:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:30:45.835629 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03daa3a6-5d76-4dff-9f77-cd3cc5e2955e None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:45.868284 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f75fe-d4e1-4530-982b-e363e8a118ca None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:45.952494 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03daa3a6-5d76-4dff-9f77-cd3cc5e2955e None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:45.962200 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ec249-1692-402d-b9a0-24088209ac9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:45.962857 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ec249-1692-402d-b9a0-24088209ac9e None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:45.963141 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ec249-1692-402d-b9a0-24088209ac9e None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:45.963805 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ec249-1692-402d-b9a0-24088209ac9e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:45.994335 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f37a3d9-5c92-4bd7-a219-add7c3bf298d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:45.995296 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f37a3d9-5c92-4bd7-a219-add7c3bf298d None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:45.995793 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f37a3d9-5c92-4bd7-a219-add7c3bf298d None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:45.996254 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f37a3d9-5c92-4bd7-a219-add7c3bf298d None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:45.997135 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f37a3d9-5c92-4bd7-a219-add7c3bf298d None admin] RBAC: Adding query filter params (name=member) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:45.997628 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f37a3d9-5c92-4bd7-a219-add7c3bf298d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:45.998845 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3ec249-1692-402d-b9a0-24088209ac9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:45.999435 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f37a3d9-5c92-4bd7-a219-add7c3bf298d None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:46.015974 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:46.016148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-6a3ec249-1692-402d-b9a0-24088209ac9e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 12:30:46.016148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:46.016148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:46.016148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:46.016148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:46.016148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:46.016148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:46.016148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:46.016148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:46.016148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:46.016148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:46.016148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:46.016148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:46.016148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:46.016148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:46.016148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:46.016958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:46.016958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:46.016958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:46.016958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:46.016958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:46.016958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:46.016958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:46.016958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:46.016958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:46.016958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:46.016958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:46.016958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:46.016958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:46.016958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:46.016958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:46.016958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:46.017775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:46.017775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:46.017775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:46.017775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:46.017775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:46.017775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:46.017775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:46.017775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:46.017775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:46.017775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:46.017775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 12:30:46.017775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:30:46.017775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 204/421] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 12:30:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:30:46.019502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 218/422] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 12:30:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:46.020803 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ec249-1692-402d-b9a0-24088209ac9e None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:46.027471 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f37a3d9-5c92-4bd7-a219-add7c3bf298d None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:46.226574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:46.241964 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ec249-1692-402d-b9a0-24088209ac9e None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:46.248246 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f37a3d9-5c92-4bd7-a219-add7c3bf298d None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:46.320272 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9274a4f-21a6-4024-aa45-83d96ca06586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:46.320904 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9274a4f-21a6-4024-aa45-83d96ca06586 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:46.321335 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9274a4f-21a6-4024-aa45-83d96ca06586 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:46.321441 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9274a4f-21a6-4024-aa45-83d96ca06586 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:46.322111 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9274a4f-21a6-4024-aa45-83d96ca06586 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:46.322414 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9274a4f-21a6-4024-aa45-83d96ca06586 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:46.323607 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71475e33-d5a1-4620-b09c-0451c071ca5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:46.324830 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71475e33-d5a1-4620-b09c-0451c071ca5a None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:46.325137 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71475e33-d5a1-4620-b09c-0451c071ca5a None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:46.325394 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71475e33-d5a1-4620-b09c-0451c071ca5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:46.336036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9274a4f-21a6-4024-aa45-83d96ca06586 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:46.361746 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 205/423] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:30:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:46.368267 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9274a4f-21a6-4024-aa45-83d96ca06586 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:46.430368 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71475e33-d5a1-4620-b09c-0451c071ca5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:46.433925 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71475e33-d5a1-4620-b09c-0451c071ca5a None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:46.437180 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:46.558806 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 219/424] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:30:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 12:30:46.567358 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71475e33-d5a1-4620-b09c-0451c071ca5a None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:46.641669 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9274a4f-21a6-4024-aa45-83d96ca06586 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:46.712097 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26010441-47f0-47b1-9cca-bcf4e16820d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:46.712738 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26010441-47f0-47b1-9cca-bcf4e16820d0 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:46.713027 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26010441-47f0-47b1-9cca-bcf4e16820d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:46.713298 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26010441-47f0-47b1-9cca-bcf4e16820d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:46.750709 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71475e33-d5a1-4620-b09c-0451c071ca5a None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:46.759678 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:46.787018 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:46.799381 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51aae4-5575-4279-a8c9-16d8946e54ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:46.800089 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51aae4-5575-4279-a8c9-16d8946e54ec None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:46.801541 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51aae4-5575-4279-a8c9-16d8946e54ec None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:46.804734 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51aae4-5575-4279-a8c9-16d8946e54ec None admin] PATH_INFO: `/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:46.806973 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51aae4-5575-4279-a8c9-16d8946e54ec None admin] RBAC: Authorizing `identity:get_project(project_id=6ee4b811a8dc4bc39e6ed0d4275b1782)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:46.807641 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51aae4-5575-4279-a8c9-16d8946e54ec None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:46.815170 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 220/425] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:30:46 2022] GET /identity/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782 => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:46.821653 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51aae4-5575-4279-a8c9-16d8946e54ec None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:46.935346 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51aae4-5575-4279-a8c9-16d8946e54ec None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:46.951253 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26010441-47f0-47b1-9cca-bcf4e16820d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:46.953196 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26010441-47f0-47b1-9cca-bcf4e16820d0 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:46.974421 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266402f1-ca47-470f-b46b-df44dd8156c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:46.975426 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266402f1-ca47-470f-b46b-df44dd8156c3 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:46.976466 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266402f1-ca47-470f-b46b-df44dd8156c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:46.976466 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266402f1-ca47-470f-b46b-df44dd8156c3 None admin] PATH_INFO: `/v3/groups/93681966bd5a46368ddcce563e694a25` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:46.977815 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266402f1-ca47-470f-b46b-df44dd8156c3 None admin] RBAC: Authorizing `identity:get_group(group_id=93681966bd5a46368ddcce563e694a25)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:46.983211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266402f1-ca47-470f-b46b-df44dd8156c3 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:46.983211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 221/426] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Oct 3 12:30:46 2022] GET /identity/v3/groups/93681966bd5a46368ddcce563e694a25 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:46.988328 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716a15b-d8ba-4cfe-9675-bc1df8cff909 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:46.988613 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716a15b-d8ba-4cfe-9675-bc1df8cff909 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:46.988888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716a15b-d8ba-4cfe-9675-bc1df8cff909 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:46.990860 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 222/427] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:46.995775 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c716a15b-d8ba-4cfe-9675-bc1df8cff909 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:47.067736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:47.117667 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 206/428] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:30:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 749 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 12:30:47.120067 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2fc1c0-c62a-4099-82a7-049d7a5d7370 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:47.120356 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2fc1c0-c62a-4099-82a7-049d7a5d7370 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:47.120632 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2fc1c0-c62a-4099-82a7-049d7a5d7370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:47.148552 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c716a15b-d8ba-4cfe-9675-bc1df8cff909 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:47.154996 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-da2fc1c0-c62a-4099-82a7-049d7a5d7370 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:47.164987 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-da2fc1c0-c62a-4099-82a7-049d7a5d7370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:47.206924 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81616cd4-7bcb-4a64-aac4-53ec90e17182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:47.207747 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81616cd4-7bcb-4a64-aac4-53ec90e17182 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:47.208127 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81616cd4-7bcb-4a64-aac4-53ec90e17182 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:47.209256 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81616cd4-7bcb-4a64-aac4-53ec90e17182 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:47.210102 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81616cd4-7bcb-4a64-aac4-53ec90e17182 None admin] RBAC: Adding query filter params (group.id=93681966bd5a46368ddcce563e694a25, role.id=47d5e2ea167241d5a27c8bbf4593eead, scope.project.id=6ee4b811a8dc4bc39e6ed0d4275b1782) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:47.210781 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81616cd4-7bcb-4a64-aac4-53ec90e17182 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:47.213497 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81616cd4-7bcb-4a64-aac4-53ec90e17182 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:47.227485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 223/429] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Oct 3 12:30:46 2022] GET /identity/v3/role_assignments?group.id=93681966bd5a46368ddcce563e694a25&scope.project.id=6ee4b811a8dc4bc39e6ed0d4275b1782&role.id=47d5e2ea167241d5a27c8bbf4593eead => generated 611 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:47.227485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81616cd4-7bcb-4a64-aac4-53ec90e17182 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:47.277497 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:47.315106 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 207/430] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:30:47.427161 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:47.431037 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81616cd4-7bcb-4a64-aac4-53ec90e17182 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:47.479342 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d9264b-4ea0-497f-9ae1-d44eefbdfc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:47.479342 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d9264b-4ea0-497f-9ae1-d44eefbdfc41 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:47.479342 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d9264b-4ea0-497f-9ae1-d44eefbdfc41 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:47.479342 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d9264b-4ea0-497f-9ae1-d44eefbdfc41 None admin] PATH_INFO: `/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:47.479737 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d9264b-4ea0-497f-9ae1-d44eefbdfc41 None admin] RBAC: Authorizing `identity:get_project(project_id=6ee4b811a8dc4bc39e6ed0d4275b1782)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:47.479737 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d9264b-4ea0-497f-9ae1-d44eefbdfc41 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:47.481833 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 224/431] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:30:47 2022] GET /identity/v3/projects/6ee4b811a8dc4bc39e6ed0d4275b1782 => generated 298 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:47.492437 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2fc1c0-c62a-4099-82a7-049d7a5d7370 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:47.626976 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2fc1c0-c62a-4099-82a7-049d7a5d7370 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:47.665267 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aad6d4-989b-41dc-8a11-c98048891fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:47.665913 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aad6d4-989b-41dc-8a11-c98048891fb9 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:47.666194 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aad6d4-989b-41dc-8a11-c98048891fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:47.666469 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aad6d4-989b-41dc-8a11-c98048891fb9 None admin] PATH_INFO: `/v3/groups/93681966bd5a46368ddcce563e694a25` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:47.667616 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aad6d4-989b-41dc-8a11-c98048891fb9 None admin] RBAC: Authorizing `identity:get_group(group_id=93681966bd5a46368ddcce563e694a25)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:47.669117 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aad6d4-989b-41dc-8a11-c98048891fb9 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:47.671087 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 208/432] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Oct 3 12:30:47 2022] GET /identity/v3/groups/93681966bd5a46368ddcce563e694a25 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:47.677040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d9264b-4ea0-497f-9ae1-d44eefbdfc41 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:47.727870 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aad6d4-989b-41dc-8a11-c98048891fb9 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:47.824097 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d9264b-4ea0-497f-9ae1-d44eefbdfc41 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:47.827214 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:47.879364 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc30f5f-c6f4-44bb-91ad-b9128d26b8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:47.879364 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc30f5f-c6f4-44bb-91ad-b9128d26b8c4 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:47.879364 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc30f5f-c6f4-44bb-91ad-b9128d26b8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:47.879364 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc30f5f-c6f4-44bb-91ad-b9128d26b8c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:47.879785 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc30f5f-c6f4-44bb-91ad-b9128d26b8c4 None admin] RBAC: Adding query filter params (group.id=93681966bd5a46368ddcce563e694a25, role.id=73b935a1ebc448899bbfc2381168d1cd, scope.project.id=6ee4b811a8dc4bc39e6ed0d4275b1782) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:47.879785 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc30f5f-c6f4-44bb-91ad-b9128d26b8c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:47.879785 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc30f5f-c6f4-44bb-91ad-b9128d26b8c4 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:47.885415 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 225/433] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Oct 3 12:30:47 2022] GET /identity/v3/role_assignments?group.id=93681966bd5a46368ddcce563e694a25&scope.project.id=6ee4b811a8dc4bc39e6ed0d4275b1782&role.id=73b935a1ebc448899bbfc2381168d1cd => generated 611 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:47.921435 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aad6d4-989b-41dc-8a11-c98048891fb9 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:47.929956 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e0d63-e6c6-46e4-b9a4-c2135ae42a37 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:47.930256 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e0d63-e6c6-46e4-b9a4-c2135ae42a37 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:47.930699 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e0d63-e6c6-46e4-b9a4-c2135ae42a37 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:47.936127 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 226/434] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:47.939028 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36602ea-3520-489b-98e3-156132602640 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:47.939326 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36602ea-3520-489b-98e3-156132602640 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:47.939747 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36602ea-3520-489b-98e3-156132602640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:47.970395 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-a36602ea-3520-489b-98e3-156132602640 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:47.971667 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-a36602ea-3520-489b-98e3-156132602640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:47.985022 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727db028-cc30-4101-83fb-fadb22434ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:47.985655 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727db028-cc30-4101-83fb-fadb22434ab9 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:47.985942 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727db028-cc30-4101-83fb-fadb22434ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:47.986214 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727db028-cc30-4101-83fb-fadb22434ab9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:48.009211 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727db028-cc30-4101-83fb-fadb22434ab9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:48.015925 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-727db028-cc30-4101-83fb-fadb22434ab9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 12:30:48.015925 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:48.015925 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:48.015925 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:48.015925 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:48.015925 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:48.015925 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:48.015925 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:48.015925 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:48.015925 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:48.015925 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:48.015925 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:48.015925 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:48.015925 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:48.015925 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:48.015925 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:48.016910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:48.016910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:48.016910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:48.016910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:48.016910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:48.016910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:48.016910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:48.016910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:48.016910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:48.016910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:48.016910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:48.016910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:48.016910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:48.016910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:48.016910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:48.016910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:48.017723 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:48.017723 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:48.017723 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:48.017723 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:48.017723 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:48.017723 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:48.017723 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:48.017723 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:48.017723 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:48.017723 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:48.017723 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 12:30:48.017723 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:30:48.017723 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 209/435] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:30:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:30:48.019362 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1842b07-41e4-47d4-8300-6065f5095312 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:48.019744 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1842b07-41e4-47d4-8300-6065f5095312 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:48.023344 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1842b07-41e4-47d4-8300-6065f5095312 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:48.023344 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 210/436] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:48.027826 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1842b07-41e4-47d4-8300-6065f5095312 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:48.030538 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:48.100286 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 227/437] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:30:48.101571 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c56403-ab2a-442a-aec9-5b5e4107262c None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:48.101719 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c56403-ab2a-442a-aec9-5b5e4107262c None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:48.101997 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c56403-ab2a-442a-aec9-5b5e4107262c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:48.138994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-06c56403-ab2a-442a-aec9-5b5e4107262c None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:48.145543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-06c56403-ab2a-442a-aec9-5b5e4107262c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:48.204621 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1842b07-41e4-47d4-8300-6065f5095312 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:48.233402 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:48.266962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 228/438] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:30:48.269288 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5380b9ad-f74a-4b79-9634-eb5d24857455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:48.269897 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380b9ad-f74a-4b79-9634-eb5d24857455 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:48.270171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380b9ad-f74a-4b79-9634-eb5d24857455 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:48.270441 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380b9ad-f74a-4b79-9634-eb5d24857455 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:48.271227 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5380b9ad-f74a-4b79-9634-eb5d24857455 None admin] RBAC: Adding query filter params (name=member) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:48.275360 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5380b9ad-f74a-4b79-9634-eb5d24857455 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:48.275360 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5380b9ad-f74a-4b79-9634-eb5d24857455 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:48.285248 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 211/439] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 12:30:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:48.295439 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c56403-ab2a-442a-aec9-5b5e4107262c None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:48.387564 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5380b9ad-f74a-4b79-9634-eb5d24857455 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:48.398612 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:48.414511 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c56403-ab2a-442a-aec9-5b5e4107262c None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:48.473633 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470fbeb7-5ea9-47dd-b2c3-fe7213292d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:48.473633 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fbeb7-5ea9-47dd-b2c3-fe7213292d32 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:48.473922 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fbeb7-5ea9-47dd-b2c3-fe7213292d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:48.474419 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fbeb7-5ea9-47dd-b2c3-fe7213292d32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:48.551294 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5380b9ad-f74a-4b79-9634-eb5d24857455 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:48.572886 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470fbeb7-5ea9-47dd-b2c3-fe7213292d32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:48.574667 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470fbeb7-5ea9-47dd-b2c3-fe7213292d32 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:48.593529 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91120d18-0939-447c-98a7-c7853d14fc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:48.594180 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91120d18-0939-447c-98a7-c7853d14fc10 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:48.594459 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91120d18-0939-447c-98a7-c7853d14fc10 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:48.594738 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91120d18-0939-447c-98a7-c7853d14fc10 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:48.606188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91120d18-0939-447c-98a7-c7853d14fc10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:48.611609 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-91120d18-0939-447c-98a7-c7853d14fc10 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 12:30:48.611609 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:48.611609 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:48.611609 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:48.611609 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:48.611609 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:48.611609 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:48.611609 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:48.611609 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:48.611609 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:48.611609 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:48.611609 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:48.611609 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:48.611609 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:48.611609 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:48.611609 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:48.612481 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:48.612481 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:48.612481 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:48.612481 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:48.612481 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:48.612481 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:48.612481 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:48.612481 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:48.612481 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:48.612481 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:48.612481 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:48.612481 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:48.612481 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:48.612481 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:48.612481 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:48.612481 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:48.613270 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:48.613270 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:48.613270 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:48.613270 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:48.613270 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:48.613270 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:48.613270 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:48.613270 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:48.613270 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:48.613270 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:48.613270 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 12:30:48.613270 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:30:48.613270 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 212/440] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 12:30:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:30:48.613926 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91120d18-0939-447c-98a7-c7853d14fc10 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:48.659376 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:48.686112 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:48.696215 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 229/441] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:30:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 12:30:48.701648 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470fbeb7-5ea9-47dd-b2c3-fe7213292d32 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:48.734401 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91120d18-0939-447c-98a7-c7853d14fc10 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:48.775666 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e3010-e4ca-40a8-b598-c91cf9649eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:48.776511 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e3010-e4ca-40a8-b598-c91cf9649eb4 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:48.776796 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e3010-e4ca-40a8-b598-c91cf9649eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:48.777076 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e3010-e4ca-40a8-b598-c91cf9649eb4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:48.790335 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176e3010-e4ca-40a8-b598-c91cf9649eb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:48.795260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-176e3010-e4ca-40a8-b598-c91cf9649eb4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:30:48.795260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:48.795260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:48.795260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:48.795260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:48.795260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:48.795260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:48.795260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:48.795260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:48.795260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:48.795260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:48.795260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:48.795260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:48.795260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:48.795260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:48.795260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:48.796165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:48.796165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:48.796165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:48.796165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:48.796165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:48.796165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:48.796165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:48.796165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:48.796165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:48.796165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:48.796165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:48.796165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:48.796165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:48.796165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:48.796165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:48.796165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:48.796953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:48.796953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:48.796953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:48.796953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:48.796953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:48.796953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:48.796953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:48.796953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:48.796953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:48.796953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:48.796953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:30:48.796953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:30:48.796953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 213/442] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 12:30:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:30:48.802217 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e3010-e4ca-40a8-b598-c91cf9649eb4 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:48.818567 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470fbeb7-5ea9-47dd-b2c3-fe7213292d32 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:48.855635 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a816e4a-4024-418f-b170-d2d030c5adc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:48.856632 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a816e4a-4024-418f-b170-d2d030c5adc9 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:48.856914 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a816e4a-4024-418f-b170-d2d030c5adc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:48.857516 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a816e4a-4024-418f-b170-d2d030c5adc9 None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:48.858176 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a816e4a-4024-418f-b170-d2d030c5adc9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:48.858480 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a816e4a-4024-418f-b170-d2d030c5adc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:48.859891 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a816e4a-4024-418f-b170-d2d030c5adc9 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:48.868795 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 230/443] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:30:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:48.869688 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a816e4a-4024-418f-b170-d2d030c5adc9 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:48.910517 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:48.921414 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e3010-e4ca-40a8-b598-c91cf9649eb4 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:48.929685 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:48.960249 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9edb7f7-65c3-4474-813a-4e2c11fb58c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:48.960881 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edb7f7-65c3-4474-813a-4e2c11fb58c3 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:48.961160 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edb7f7-65c3-4474-813a-4e2c11fb58c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:48.961431 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edb7f7-65c3-4474-813a-4e2c11fb58c3 None admin] PATH_INFO: `/v3/projects/c974a924a68947af9e8abc10c2614ae7` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:48.962728 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9edb7f7-65c3-4474-813a-4e2c11fb58c3 None admin] RBAC: Authorizing `identity:get_project(project_id=c974a924a68947af9e8abc10c2614ae7)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:48.964246 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9edb7f7-65c3-4474-813a-4e2c11fb58c3 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:48.970856 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 214/444] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:30:48 2022] GET /identity/v3/projects/c974a924a68947af9e8abc10c2614ae7 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:48.971763 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9edb7f7-65c3-4474-813a-4e2c11fb58c3 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:48.982197 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a816e4a-4024-418f-b170-d2d030c5adc9 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:49.020398 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43090596-7ca6-411e-9fec-e29a3db73677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:49.021016 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43090596-7ca6-411e-9fec-e29a3db73677 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:49.021364 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43090596-7ca6-411e-9fec-e29a3db73677 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:49.022164 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43090596-7ca6-411e-9fec-e29a3db73677 None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:49.022458 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43090596-7ca6-411e-9fec-e29a3db73677 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:49.022787 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43090596-7ca6-411e-9fec-e29a3db73677 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:49.024188 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43090596-7ca6-411e-9fec-e29a3db73677 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:49.032721 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 231/445] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:30:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:49.033529 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43090596-7ca6-411e-9fec-e29a3db73677 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:49.089721 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9edb7f7-65c3-4474-813a-4e2c11fb58c3 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:49.126062 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0b2ae-1cbd-4aa3-8a4c-f07dcf85c36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:49.126934 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0b2ae-1cbd-4aa3-8a4c-f07dcf85c36d None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:49.127384 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0b2ae-1cbd-4aa3-8a4c-f07dcf85c36d None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:49.127660 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0b2ae-1cbd-4aa3-8a4c-f07dcf85c36d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:49.148902 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43090596-7ca6-411e-9fec-e29a3db73677 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:49.160985 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:49.166199 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:49.184428 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba6ed4-29b2-46c7-b284-dd9c9ac980e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:49.185063 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba6ed4-29b2-46c7-b284-dd9c9ac980e0 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:49.185397 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba6ed4-29b2-46c7-b284-dd9c9ac980e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:49.185674 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba6ed4-29b2-46c7-b284-dd9c9ac980e0 None admin] PATH_INFO: `/v3/groups/93681966bd5a46368ddcce563e694a25` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:49.186903 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba6ed4-29b2-46c7-b284-dd9c9ac980e0 None admin] RBAC: Authorizing `identity:get_group(group_id=93681966bd5a46368ddcce563e694a25)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:49.188294 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba6ed4-29b2-46c7-b284-dd9c9ac980e0 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:49.189885 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 232/446] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Oct 3 12:30:49 2022] GET /identity/v3/groups/93681966bd5a46368ddcce563e694a25 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:49.190623 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba6ed4-29b2-46c7-b284-dd9c9ac980e0 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:49.210032 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c0b2ae-1cbd-4aa3-8a4c-f07dcf85c36d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:49.211642 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c0b2ae-1cbd-4aa3-8a4c-f07dcf85c36d None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:49.307289 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba6ed4-29b2-46c7-b284-dd9c9ac980e0 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:49.327860 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 215/447] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:30:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 12:30:49.328713 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0b2ae-1cbd-4aa3-8a4c-f07dcf85c36d None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:49.348659 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8c2227-3770-44c1-a820-0bb7d441450b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:49.349253 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c2227-3770-44c1-a820-0bb7d441450b None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:49.349604 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c2227-3770-44c1-a820-0bb7d441450b None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:49.349871 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c2227-3770-44c1-a820-0bb7d441450b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:49.401201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:49.404723 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:49.426669 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8c2227-3770-44c1-a820-0bb7d441450b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:49.428580 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8c2227-3770-44c1-a820-0bb7d441450b None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:49.447030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0b2ae-1cbd-4aa3-8a4c-f07dcf85c36d None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:49.484649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1682ddf-9a03-4ba6-90ab-667e4cdf57e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:49.485269 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1682ddf-9a03-4ba6-90ab-667e4cdf57e0 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:49.485551 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1682ddf-9a03-4ba6-90ab-667e4cdf57e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:49.485824 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1682ddf-9a03-4ba6-90ab-667e4cdf57e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:49.486542 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1682ddf-9a03-4ba6-90ab-667e4cdf57e0 None admin] RBAC: Adding query filter params (group.id=93681966bd5a46368ddcce563e694a25, role.id=47d5e2ea167241d5a27c8bbf4593eead, scope.project.id=c974a924a68947af9e8abc10c2614ae7) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:49.486850 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1682ddf-9a03-4ba6-90ab-667e4cdf57e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:49.488462 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1682ddf-9a03-4ba6-90ab-667e4cdf57e0 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:49.494658 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 216/448] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Oct 3 12:30:49 2022] GET /identity/v3/role_assignments?group.id=93681966bd5a46368ddcce563e694a25&scope.project.id=c974a924a68947af9e8abc10c2614ae7&role.id=47d5e2ea167241d5a27c8bbf4593eead => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:49.495488 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1682ddf-9a03-4ba6-90ab-667e4cdf57e0 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:49.545334 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 233/449] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:30:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 12:30:49.551260 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8c2227-3770-44c1-a820-0bb7d441450b None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:49.622247 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1682ddf-9a03-4ba6-90ab-667e4cdf57e0 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:49.657718 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:49.670613 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:49.675953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21035a6-9fc5-4b54-946d-f0e8e9562d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:49.676605 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21035a6-9fc5-4b54-946d-f0e8e9562d7a None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:49.676883 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21035a6-9fc5-4b54-946d-f0e8e9562d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:49.677153 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21035a6-9fc5-4b54-946d-f0e8e9562d7a None admin] PATH_INFO: `/v3/projects/c974a924a68947af9e8abc10c2614ae7` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:49.680103 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21035a6-9fc5-4b54-946d-f0e8e9562d7a None admin] RBAC: Authorizing `identity:get_project(project_id=c974a924a68947af9e8abc10c2614ae7)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:49.681394 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21035a6-9fc5-4b54-946d-f0e8e9562d7a None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:49.687571 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 217/450] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:30:49 2022] GET /identity/v3/projects/c974a924a68947af9e8abc10c2614ae7 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:49.689811 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21035a6-9fc5-4b54-946d-f0e8e9562d7a None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:49.692031 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8c2227-3770-44c1-a820-0bb7d441450b None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:49.748211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a1544d-5bba-4d36-9f83-f9695c7bf1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:49.749296 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a1544d-5bba-4d36-9f83-f9695c7bf1b9 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:49.749296 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a1544d-5bba-4d36-9f83-f9695c7bf1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:49.749461 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a1544d-5bba-4d36-9f83-f9695c7bf1b9 None admin] PATH_INFO: `/v3/projects/912de75d08d24473bf98cc746c209242` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:49.750551 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a1544d-5bba-4d36-9f83-f9695c7bf1b9 None admin] RBAC: Authorizing `identity:get_project(project_id=912de75d08d24473bf98cc746c209242)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:49.751974 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a1544d-5bba-4d36-9f83-f9695c7bf1b9 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:49.753628 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 234/451] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:30:49 2022] GET /identity/v3/projects/912de75d08d24473bf98cc746c209242 => generated 344 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:30:49.761520 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a1544d-5bba-4d36-9f83-f9695c7bf1b9 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:49.853947 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21035a6-9fc5-4b54-946d-f0e8e9562d7a None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:49.905079 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca67b10a-eb74-4204-9840-873516957054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:49.905079 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67b10a-eb74-4204-9840-873516957054 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:49.905079 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67b10a-eb74-4204-9840-873516957054 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:49.905079 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67b10a-eb74-4204-9840-873516957054 None admin] PATH_INFO: `/v3/groups/93681966bd5a46368ddcce563e694a25` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:49.905539 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca67b10a-eb74-4204-9840-873516957054 None admin] RBAC: Authorizing `identity:get_group(group_id=93681966bd5a46368ddcce563e694a25)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:49.906975 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca67b10a-eb74-4204-9840-873516957054 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:49.908669 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 218/452] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Oct 3 12:30:49 2022] GET /identity/v3/groups/93681966bd5a46368ddcce563e694a25 => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:49.914600 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca67b10a-eb74-4204-9840-873516957054 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:49.919740 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a1544d-5bba-4d36-9f83-f9695c7bf1b9 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:49.958403 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd62e52-2e5c-40f3-ad7f-a231661579ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:49.959443 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd62e52-2e5c-40f3-ad7f-a231661579ce None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:49.959443 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd62e52-2e5c-40f3-ad7f-a231661579ce None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:49.959702 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd62e52-2e5c-40f3-ad7f-a231661579ce None admin] PATH_INFO: `/v3/groups/26b4b111512144a28363e2bf8784330e` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:49.960885 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd62e52-2e5c-40f3-ad7f-a231661579ce None admin] RBAC: Authorizing `identity:get_group(group_id=26b4b111512144a28363e2bf8784330e)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:49.962305 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd62e52-2e5c-40f3-ad7f-a231661579ce None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:49.963971 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 235/453] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Oct 3 12:30:49 2022] GET /identity/v3/groups/26b4b111512144a28363e2bf8784330e => generated 229 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:49.970000 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd62e52-2e5c-40f3-ad7f-a231661579ce None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:49.986814 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:49.995960 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:50.029629 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca67b10a-eb74-4204-9840-873516957054 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:50.065701 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0325a9b-de5f-4ae8-b741-72e93cc43289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:50.066345 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0325a9b-de5f-4ae8-b741-72e93cc43289 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:50.066654 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0325a9b-de5f-4ae8-b741-72e93cc43289 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:50.066926 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0325a9b-de5f-4ae8-b741-72e93cc43289 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:50.067687 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0325a9b-de5f-4ae8-b741-72e93cc43289 None admin] RBAC: Adding query filter params (group.id=93681966bd5a46368ddcce563e694a25, role.id=73b935a1ebc448899bbfc2381168d1cd, scope.project.id=c974a924a68947af9e8abc10c2614ae7) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:50.068110 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0325a9b-de5f-4ae8-b741-72e93cc43289 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:50.070161 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0325a9b-de5f-4ae8-b741-72e93cc43289 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:50.075655 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd62e52-2e5c-40f3-ad7f-a231661579ce None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:50.079511 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 219/454] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Oct 3 12:30:49 2022] GET /identity/v3/role_assignments?group.id=93681966bd5a46368ddcce563e694a25&scope.project.id=c974a924a68947af9e8abc10c2614ae7&role.id=73b935a1ebc448899bbfc2381168d1cd => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:50.118197 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e7c986-b463-4753-80ef-2a4d989de1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:50.118939 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e7c986-b463-4753-80ef-2a4d989de1f4 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:50.119258 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e7c986-b463-4753-80ef-2a4d989de1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:50.119652 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e7c986-b463-4753-80ef-2a4d989de1f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:50.120407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e7c986-b463-4753-80ef-2a4d989de1f4 None admin] RBAC: Adding query filter params (group.id=26b4b111512144a28363e2bf8784330e, role.id=7933c45dcdd54ef0a98018ba8a8bddaa, scope.project.id=912de75d08d24473bf98cc746c209242) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:50.120716 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e7c986-b463-4753-80ef-2a4d989de1f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:50.122322 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e7c986-b463-4753-80ef-2a4d989de1f4 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:50.130333 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 236/455] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Oct 3 12:30:49 2022] GET /identity/v3/role_assignments?group.id=26b4b111512144a28363e2bf8784330e&scope.project.id=912de75d08d24473bf98cc746c209242&role.id=7933c45dcdd54ef0a98018ba8a8bddaa => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:30:53.414543 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfec125-4762-4255-9790-2591d2667046 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:53.414543 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfec125-4762-4255-9790-2591d2667046 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:53.414543 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfec125-4762-4255-9790-2591d2667046 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:53.416290 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 220/456] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:53.421278 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c25a7c-ee12-45e3-9791-4dc2c99b67c9 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:53.421675 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c25a7c-ee12-45e3-9791-4dc2c99b67c9 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:53.421857 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c25a7c-ee12-45e3-9791-4dc2c99b67c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:53.452520 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-b2c25a7c-ee12-45e3-9791-4dc2c99b67c9 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:53.454118 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-b2c25a7c-ee12-45e3-9791-4dc2c99b67c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:53.565957 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 237/457] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:30:53.589306 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9938b4-fd76-46dc-9ab9-92ab5a4b2bb6 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:53.589930 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9938b4-fd76-46dc-9ab9-92ab5a4b2bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:53.590407 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9938b4-fd76-46dc-9ab9-92ab5a4b2bb6 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:53.592685 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 221/458] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:53.598021 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697fe9d-2c32-46d0-8136-b96453824312 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:53.598541 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697fe9d-2c32-46d0-8136-b96453824312 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:53.599034 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697fe9d-2c32-46d0-8136-b96453824312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:53.629266 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-a697fe9d-2c32-46d0-8136-b96453824312 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:53.636773 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-a697fe9d-2c32-46d0-8136-b96453824312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:53.650429 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:53.787702 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e2f7c-468c-418b-ab39-24ea7232c482 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:53.788447 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e2f7c-468c-418b-ab39-24ea7232c482 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:53.788951 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e2f7c-468c-418b-ab39-24ea7232c482 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:53.791066 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 222/459] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:53.798800 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89945228-59ed-4bc1-ba2b-832a0862777d None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:53.798800 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89945228-59ed-4bc1-ba2b-832a0862777d None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:53.798800 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89945228-59ed-4bc1-ba2b-832a0862777d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:53.803558 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 238/460] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:30:53.852720 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-89945228-59ed-4bc1-ba2b-832a0862777d None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:53.854246 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-89945228-59ed-4bc1-ba2b-832a0862777d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:53.992964 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 223/461] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:30:54.032463 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a697fe9d-2c32-46d0-8136-b96453824312 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:54.202257 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89945228-59ed-4bc1-ba2b-832a0862777d None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:54.342812 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a697fe9d-2c32-46d0-8136-b96453824312 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:54.366165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:54.415808 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41f0886-5bfa-4275-a246-2c3b5ec25b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:54.416838 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f0886-5bfa-4275-a246-2c3b5ec25b9f None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:54.417317 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f0886-5bfa-4275-a246-2c3b5ec25b9f None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:54.418126 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f0886-5bfa-4275-a246-2c3b5ec25b9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:54.440105 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41f0886-5bfa-4275-a246-2c3b5ec25b9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:54.442501 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41f0886-5bfa-4275-a246-2c3b5ec25b9f None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:54.445954 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89945228-59ed-4bc1-ba2b-832a0862777d None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:54.451047 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:54.472079 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-d41f0886-5bfa-4275-a246-2c3b5ec25b9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:30:54.472079 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:54.472079 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:54.472079 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:54.472079 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:54.472079 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:54.472079 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:54.472079 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:54.472079 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:54.472079 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:54.472079 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:54.472079 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:54.472079 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:30:54.472079 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:30:54.472079 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:30:54.472079 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:30:54.473038 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:54.473038 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:54.473038 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:54.473038 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:54.473038 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:54.473038 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:54.473038 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:54.473038 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:54.473038 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:54.473038 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:54.473038 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:54.473038 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:54.473038 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:54.473038 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:54.473038 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:54.473038 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:54.473038 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:30:54.473909 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:30:54.473909 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:30:54.473909 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:30:54.479948 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 239/462] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:30:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 448 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:30:54.481140 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41f0886-5bfa-4275-a246-2c3b5ec25b9f None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:54.550987 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82db3dbe-765f-44b3-a428-ec0b1b99df05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:54.551906 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db3dbe-765f-44b3-a428-ec0b1b99df05 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:54.553168 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db3dbe-765f-44b3-a428-ec0b1b99df05 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:54.553868 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db3dbe-765f-44b3-a428-ec0b1b99df05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:54.573810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82db3dbe-765f-44b3-a428-ec0b1b99df05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:54.575710 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82db3dbe-765f-44b3-a428-ec0b1b99df05 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:54.604095 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-82db3dbe-765f-44b3-a428-ec0b1b99df05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:30:54.604095 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:54.604095 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:54.604095 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:54.604095 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:54.604095 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:54.604095 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:54.604095 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:54.604095 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:54.604095 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:54.604095 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:54.604095 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:54.604095 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:30:54.604095 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:30:54.604095 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:30:54.604095 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:30:54.605139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:54.605139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:54.605139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:54.605139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:54.605139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:54.605139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:54.605139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:54.605139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:54.605139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:54.605139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:54.605139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:54.605139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:54.605139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:54.605139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:54.605139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:54.605139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:54.605139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:30:54.606005 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:30:54.606005 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:30:54.606005 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:30:54.606005 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 224/463] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:30:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 404 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:30:54.606555 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82db3dbe-765f-44b3-a428-ec0b1b99df05 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:54.608254 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41f0886-5bfa-4275-a246-2c3b5ec25b9f None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:54.659514 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e7f8c-6e7c-496e-a5e5-9e609e8ae5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:54.660597 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e7f8c-6e7c-496e-a5e5-9e609e8ae5d3 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:54.661109 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e7f8c-6e7c-496e-a5e5-9e609e8ae5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:54.661627 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e7f8c-6e7c-496e-a5e5-9e609e8ae5d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:54.674684 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e7f8c-6e7c-496e-a5e5-9e609e8ae5d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:54.676568 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e7f8c-6e7c-496e-a5e5-9e609e8ae5d3 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:54.692713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-883e7f8c-6e7c-496e-a5e5-9e609e8ae5d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:30:54.692713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:54.692713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:54.692713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:54.692713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:54.692713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:54.692713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:54.692713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:54.692713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:54.692713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:54.692713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:54.692713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:54.692713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:30:54.692713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:30:54.692713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:30:54.692713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:30:54.694684 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:54.694684 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:54.694684 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:54.694684 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:54.694684 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:54.694684 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:54.694684 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:54.694684 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:54.694684 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:54.694684 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:54.694684 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:54.694684 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:54.694684 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:54.694684 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:54.694684 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:54.694684 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:54.694684 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:30:54.695679 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:30:54.695679 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:30:54.695679 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:30:54.696443 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 240/464] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:30:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:30:54.697821 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703df26-f80a-4236-a8f4-6892bfcea14d None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:54.698107 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703df26-f80a-4236-a8f4-6892bfcea14d None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:54.698380 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703df26-f80a-4236-a8f4-6892bfcea14d None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:54.699908 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 241/465] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:54.700960 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703df26-f80a-4236-a8f4-6892bfcea14d None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:54.752814 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82db3dbe-765f-44b3-a428-ec0b1b99df05 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:54.762138 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:54.764328 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:54.788589 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d379e3-cddf-4a24-92a7-cc2e068541d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:54.789417 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d379e3-cddf-4a24-92a7-cc2e068541d2 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:54.789695 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d379e3-cddf-4a24-92a7-cc2e068541d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:54.790043 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d379e3-cddf-4a24-92a7-cc2e068541d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:54.791460 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d379e3-cddf-4a24-92a7-cc2e068541d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:54.791460 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d379e3-cddf-4a24-92a7-cc2e068541d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:54.792670 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d379e3-cddf-4a24-92a7-cc2e068541d2 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:54.807590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 225/466] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:54.809065 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d379e3-cddf-4a24-92a7-cc2e068541d2 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:54.822889 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703df26-f80a-4236-a8f4-6892bfcea14d None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:54.864381 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e27a1-a4d9-4dc4-af62-a1fa39f0d51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:54.865650 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e27a1-a4d9-4dc4-af62-a1fa39f0d51d None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:54.866120 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e27a1-a4d9-4dc4-af62-a1fa39f0d51d None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:54.867398 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e27a1-a4d9-4dc4-af62-a1fa39f0d51d None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:54.867398 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472e27a1-a4d9-4dc4-af62-a1fa39f0d51d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:54.867398 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472e27a1-a4d9-4dc4-af62-a1fa39f0d51d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:54.868771 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472e27a1-a4d9-4dc4-af62-a1fa39f0d51d None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:54.886999 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 242/467] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:54.886999 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe6dd7-aec2-456d-9a68-63ee2cb9d6ca None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:54.886999 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe6dd7-aec2-456d-9a68-63ee2cb9d6ca None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:54.886999 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe6dd7-aec2-456d-9a68-63ee2cb9d6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:54.903849 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-3fbe6dd7-aec2-456d-9a68-63ee2cb9d6ca None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:54.905080 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-3fbe6dd7-aec2-456d-9a68-63ee2cb9d6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:54.931440 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d379e3-cddf-4a24-92a7-cc2e068541d2 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:54.976615 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b5f53a-f85b-4b8f-875d-64812ebc1c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:54.977314 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5f53a-f85b-4b8f-875d-64812ebc1c26 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:54.977663 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5f53a-f85b-4b8f-875d-64812ebc1c26 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:54.977940 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5f53a-f85b-4b8f-875d-64812ebc1c26 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:54.978579 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b5f53a-f85b-4b8f-875d-64812ebc1c26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:54.978874 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b5f53a-f85b-4b8f-875d-64812ebc1c26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:54.980252 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b5f53a-f85b-4b8f-875d-64812ebc1c26 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:54.982165 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 243/468] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:30:54.983528 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69bb360-29ab-451a-8ab2-c24b8308f123 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:54.983818 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69bb360-29ab-451a-8ab2-c24b8308f123 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:54.984079 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69bb360-29ab-451a-8ab2-c24b8308f123 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:54.989501 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 244/469] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:54.990888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69bb360-29ab-451a-8ab2-c24b8308f123 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:54.995362 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 226/470] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:54.995362 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b5f53a-f85b-4b8f-875d-64812ebc1c26 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:55.030459 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:55.064947 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:55.124042 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b5f53a-f85b-4b8f-875d-64812ebc1c26 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:55.162508 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a5395-82e7-443a-a50e-25357ac32a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:55.163118 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a5395-82e7-443a-a50e-25357ac32a1d None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:55.163443 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a5395-82e7-443a-a50e-25357ac32a1d None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:55.163707 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a5395-82e7-443a-a50e-25357ac32a1d None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:55.164338 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2a5395-82e7-443a-a50e-25357ac32a1d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:55.165757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2a5395-82e7-443a-a50e-25357ac32a1d None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:55.176090 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69bb360-29ab-451a-8ab2-c24b8308f123 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:55.211758 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d409b-a31b-4fd2-8ed6-85aaf5ec4c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:55.212612 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d409b-a31b-4fd2-8ed6-85aaf5ec4c1a None admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:55.213157 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d409b-a31b-4fd2-8ed6-85aaf5ec4c1a None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:55.213624 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d409b-a31b-4fd2-8ed6-85aaf5ec4c1a None admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:55.214313 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d409b-a31b-4fd2-8ed6-85aaf5ec4c1a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:55.215705 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d409b-a31b-4fd2-8ed6-85aaf5ec4c1a None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:55.285748 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 227/471] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Oct 3 12:30:54 2022] POST /identity/v3/users => generated 248 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:30:55.287259 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce86bd-f151-4ebd-b27f-6592f69fedf0 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:55.287622 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce86bd-f151-4ebd-b27f-6592f69fedf0 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:55.287901 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce86bd-f151-4ebd-b27f-6592f69fedf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:55.316396 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-dcce86bd-f151-4ebd-b27f-6592f69fedf0 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:55.323392 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-dcce86bd-f151-4ebd-b27f-6592f69fedf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:55.324053 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 245/472] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Oct 3 12:30:54 2022] POST /identity/v3/users => generated 246 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:30:55.325052 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d409b-a31b-4fd2-8ed6-85aaf5ec4c1a None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:55.527252 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:55.535209 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d409b-a31b-4fd2-8ed6-85aaf5ec4c1a None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:55.544873 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 228/473] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:30:55.545549 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcce86bd-f151-4ebd-b27f-6592f69fedf0 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:55.569049 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:55.602348 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf2e17-9830-4f6f-b43b-833f609fa88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:55.604124 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf2e17-9830-4f6f-b43b-833f609fa88a None admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:55.604586 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf2e17-9830-4f6f-b43b-833f609fa88a None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:55.605039 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf2e17-9830-4f6f-b43b-833f609fa88a None admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:55.605868 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cf2e17-9830-4f6f-b43b-833f609fa88a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:55.607507 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cf2e17-9830-4f6f-b43b-833f609fa88a None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:55.715649 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 246/474] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Oct 3 12:30:55 2022] POST /identity/v3/users => generated 248 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:30:55.853537 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcce86bd-f151-4ebd-b27f-6592f69fedf0 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:55.919234 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c545d7f-85a9-421e-a2c6-9740c25d117d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:55.919931 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c545d7f-85a9-421e-a2c6-9740c25d117d None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:55.920211 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c545d7f-85a9-421e-a2c6-9740c25d117d None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:55.920481 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c545d7f-85a9-421e-a2c6-9740c25d117d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:55.930681 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf2e17-9830-4f6f-b43b-833f609fa88a None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:55.935730 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c545d7f-85a9-421e-a2c6-9740c25d117d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:55.937202 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c545d7f-85a9-421e-a2c6-9740c25d117d None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:55.956384 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-2c545d7f-85a9-421e-a2c6-9740c25d117d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:30:55.956384 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:55.956384 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:55.956384 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:55.956384 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:55.956384 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:55.956384 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:55.956384 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:55.956384 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:55.956384 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:55.956384 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:55.956384 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:55.956384 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:30:55.956384 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:30:55.956384 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:30:55.956384 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:30:55.957337 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:55.957337 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:55.957337 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:55.957337 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:55.957337 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:55.957337 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:55.957337 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:55.957337 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:55.957337 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:55.957337 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:55.957337 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:55.957337 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:55.957337 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:55.957337 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:55.957337 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:55.957337 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:55.957337 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:30:55.958160 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:30:55.958160 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:30:55.958160 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:30:55.958160 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 229/475] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:30:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 412 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:30:55.964359 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c545d7f-85a9-421e-a2c6-9740c25d117d None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:56.036032 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:56.062189 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf2e17-9830-4f6f-b43b-833f609fa88a None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:56.073655 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c545d7f-85a9-421e-a2c6-9740c25d117d None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:56.104365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce10e3-7448-4653-b2a5-4fdd5a91b0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:56.105196 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce10e3-7448-4653-b2a5-4fdd5a91b0ba None admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:56.105613 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce10e3-7448-4653-b2a5-4fdd5a91b0ba None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:56.106045 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce10e3-7448-4653-b2a5-4fdd5a91b0ba None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:56.106955 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce10e3-7448-4653-b2a5-4fdd5a91b0ba None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:56.108592 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce10e3-7448-4653-b2a5-4fdd5a91b0ba None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:56.117529 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2736c16b-5e10-4511-89e7-9daf1d3bcb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:56.118250 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736c16b-5e10-4511-89e7-9daf1d3bcb1f None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:56.118592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736c16b-5e10-4511-89e7-9daf1d3bcb1f None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:56.118939 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736c16b-5e10-4511-89e7-9daf1d3bcb1f None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:56.120311 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2736c16b-5e10-4511-89e7-9daf1d3bcb1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:56.120311 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2736c16b-5e10-4511-89e7-9daf1d3bcb1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:56.121416 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2736c16b-5e10-4511-89e7-9daf1d3bcb1f None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:56.134119 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 230/476] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:30:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:56.139788 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2736c16b-5e10-4511-89e7-9daf1d3bcb1f None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:56.148543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.sql.core [None req-7cce10e3-7448-4653-b2a5-4fdd5a91b0ba None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 03 12:30:56.148543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 03 12:30:56.148543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [parameters: {'id': '3cf3974f0ea2402296048713430ec274', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 03 12:30:56.148543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81806) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 03 12:30:56.169697 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-7cce10e3-7448-4653-b2a5-4fdd5a91b0ba None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 03 12:30:56.169697 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:56.169697 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 03 12:30:56.169697 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 03 12:30:56.169697 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 03 12:30:56.169697 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 03 12:30:56.169697 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 03 12:30:56.169697 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application result = self._query(query) Oct 03 12:30:56.169697 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 03 12:30:56.169697 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application conn.query(q) Oct 03 12:30:56.169697 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 03 12:30:56.169697 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 03 12:30:56.169697 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 03 12:30:56.169697 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application result.read() Oct 03 12:30:56.169697 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 03 12:30:56.170707 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 03 12:30:56.170707 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 03 12:30:56.170707 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 03 12:30:56.170707 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 03 12:30:56.170707 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 03 12:30:56.170707 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 03 12:30:56.170707 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 03 12:30:56.170707 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 03 12:30:56.170707 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:30:56.170707 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 03 12:30:56.170707 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:30:56.170707 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:56.170707 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 03 12:30:56.170707 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 03 12:30:56.170707 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 03 12:30:56.170707 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return ref.to_dict() Oct 03 12:30:56.170707 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 03 12:30:56.170707 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application next(self.gen) Oct 03 12:30:56.171979 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 03 12:30:56.171979 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application yield resource Oct 03 12:30:56.171979 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 03 12:30:56.171979 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application next(self.gen) Oct 03 12:30:56.171979 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 03 12:30:56.171979 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application self.session.rollback() Oct 03 12:30:56.171979 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 03 12:30:56.171979 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application self.force_reraise() Oct 03 12:30:56.171979 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 03 12:30:56.171979 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise self.value Oct 03 12:30:56.171979 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 03 12:30:56.171979 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 03 12:30:56.171979 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 03 12:30:56.171979 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application session.commit() Oct 03 12:30:56.171979 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 03 12:30:56.171979 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 03 12:30:56.171979 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 03 12:30:56.172839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application self._prepare_impl() Oct 03 12:30:56.172839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 03 12:30:56.172839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application self.session.flush() Oct 03 12:30:56.172839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 03 12:30:56.172839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application self._flush(objects) Oct 03 12:30:56.172839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 03 12:30:56.172839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 03 12:30:56.172839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 03 12:30:56.172839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application compat.raise_( Oct 03 12:30:56.172839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 03 12:30:56.172839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception Oct 03 12:30:56.172839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 03 12:30:56.172839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application flush_context.execute() Oct 03 12:30:56.172839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 03 12:30:56.172839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rec.execute(self) Oct 03 12:30:56.172839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 03 12:30:56.172839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 03 12:30:56.173733 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 03 12:30:56.173733 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 03 12:30:56.173733 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 03 12:30:56.173733 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 03 12:30:56.173733 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 03 12:30:56.173733 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 03 12:30:56.173733 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 03 12:30:56.173733 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 03 12:30:56.173733 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 03 12:30:56.173733 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 03 12:30:56.173733 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 03 12:30:56.173733 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 03 12:30:56.173733 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 03 12:30:56.173733 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 03 12:30:56.173733 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 03 12:30:56.173733 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception Oct 03 12:30:56.174517 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 03 12:30:56.174517 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 03 12:30:56.174517 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 03 12:30:56.174517 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 03 12:30:56.174517 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 03 12:30:56.174517 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application result = self._query(query) Oct 03 12:30:56.174517 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 03 12:30:56.174517 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application conn.query(q) Oct 03 12:30:56.174517 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 03 12:30:56.174517 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 03 12:30:56.174517 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 03 12:30:56.174517 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application result.read() Oct 03 12:30:56.174517 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 03 12:30:56.174517 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 03 12:30:56.174517 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 03 12:30:56.174517 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 03 12:30:56.178525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 03 12:30:56.178525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 03 12:30:56.178525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 03 12:30:56.178525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 03 12:30:56.178525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 03 12:30:56.178525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 03 12:30:56.178525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [parameters: {'id': '3cf3974f0ea2402296048713430ec274', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 03 12:30:56.178525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 03 12:30:56.178525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:30:56.178525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 12:30:56.178525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:30:56.178525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:56.178525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:56.178525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:56.178525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:56.178525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:56.180216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:56.180216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:56.180216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:56.180216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:56.180216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:56.180216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:56.180216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 03 12:30:56.180216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 03 12:30:56.180216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:56.180216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:56.180216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 03 12:30:56.180216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 03 12:30:56.180216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 03 12:30:56.180216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 03 12:30:56.180216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 03 12:30:56.180216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:30:56.184920 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 247/477] 10.222.0.33 () {60 vars in 1131 bytes} [Mon Oct 3 12:30:55 2022] POST /identity/v3/roles => generated 144 bytes in 259 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:30:56.192811 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce10e3-7448-4653-b2a5-4fdd5a91b0ba None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:56.216636 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:56.319497 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2736c16b-5e10-4511-89e7-9daf1d3bcb1f None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:56.368394 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce10e3-7448-4653-b2a5-4fdd5a91b0ba None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:56.371603 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:56.390260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cc814c-8c55-4464-9edd-9969d63ec6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:56.390881 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cc814c-8c55-4464-9edd-9969d63ec6fe None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:56.391152 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cc814c-8c55-4464-9edd-9969d63ec6fe None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:56.391517 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cc814c-8c55-4464-9edd-9969d63ec6fe None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:56.392168 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cc814c-8c55-4464-9edd-9969d63ec6fe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:56.396420 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cc814c-8c55-4464-9edd-9969d63ec6fe None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:56.432146 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8805ba-d248-4f8c-9ca8-da5a4390792b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:56.432882 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8805ba-d248-4f8c-9ca8-da5a4390792b None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:56.433275 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8805ba-d248-4f8c-9ca8-da5a4390792b None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:56.433562 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8805ba-d248-4f8c-9ca8-da5a4390792b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:56.446395 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8805ba-d248-4f8c-9ca8-da5a4390792b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:56.451838 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-9d8805ba-d248-4f8c-9ca8-da5a4390792b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 12:30:56.451838 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:56.451838 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:56.451838 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:56.451838 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:56.451838 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:56.451838 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:56.451838 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:56.451838 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:56.451838 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:56.451838 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:56.451838 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:56.451838 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:56.451838 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:56.451838 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:56.451838 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:56.453532 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:56.453532 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:56.453532 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:56.453532 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:56.453532 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:56.453532 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:56.453532 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:56.453532 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:56.453532 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:56.453532 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:56.453532 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:56.453532 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:56.453532 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:56.453532 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:56.453532 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:56.453532 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:56.454522 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:56.454522 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:56.454522 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:56.454522 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:56.454522 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:56.454522 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:56.454522 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:56.454522 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:56.454522 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:56.454522 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:56.454522 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 12:30:56.454522 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:30:56.454522 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 248/478] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 12:30:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:30:56.462464 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8805ba-d248-4f8c-9ca8-da5a4390792b None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:56.494122 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 231/479] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Oct 3 12:30:56 2022] POST /identity/v3/users => generated 249 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:30:56.585839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:56.620957 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8805ba-d248-4f8c-9ca8-da5a4390792b None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:56.668277 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be8640-234c-4c36-b3cd-13d460387944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:56.668907 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be8640-234c-4c36-b3cd-13d460387944 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:56.669185 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be8640-234c-4c36-b3cd-13d460387944 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:56.669456 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be8640-234c-4c36-b3cd-13d460387944 None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:56.670136 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08be8640-234c-4c36-b3cd-13d460387944 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:56.670440 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08be8640-234c-4c36-b3cd-13d460387944 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:56.671932 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08be8640-234c-4c36-b3cd-13d460387944 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:56.687665 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 249/480] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:30:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:30:58.012412 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fda73f-a869-4823-bdf7-86a52fccbf46 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:58.012412 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fda73f-a869-4823-bdf7-86a52fccbf46 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:58.012412 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fda73f-a869-4823-bdf7-86a52fccbf46 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:58.013497 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 232/481] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:58.019071 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87804504-46b0-4b7e-9213-a2b8ab37aa7c None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:58.019914 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87804504-46b0-4b7e-9213-a2b8ab37aa7c None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:58.020406 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87804504-46b0-4b7e-9213-a2b8ab37aa7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:58.042723 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8ea94-7265-4f2a-b864-0dcd317a63d7 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:58.043508 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8ea94-7265-4f2a-b864-0dcd317a63d7 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:58.043978 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8ea94-7265-4f2a-b864-0dcd317a63d7 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:58.045490 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 233/482] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 12:30:58.049238 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-87804504-46b0-4b7e-9213-a2b8ab37aa7c None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:58.050591 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12324048-6ed0-4702-b78d-15ea5d0de0f7 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:58.051064 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12324048-6ed0-4702-b78d-15ea5d0de0f7 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:58.051551 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12324048-6ed0-4702-b78d-15ea5d0de0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:58.058407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-87804504-46b0-4b7e-9213-a2b8ab37aa7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:58.083342 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-12324048-6ed0-4702-b78d-15ea5d0de0f7 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:58.086785 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-12324048-6ed0-4702-b78d-15ea5d0de0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:58.200926 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 250/483] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 12:30:58.221336 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 234/484] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 12:30:58.548253 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87804504-46b0-4b7e-9213-a2b8ab37aa7c None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:58.578523 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12324048-6ed0-4702-b78d-15ea5d0de0f7 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:58.631777 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:58.697331 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87804504-46b0-4b7e-9213-a2b8ab37aa7c None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:58.717112 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12324048-6ed0-4702-b78d-15ea5d0de0f7 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:58.734625 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:58.768376 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b8fda8-388e-4cf5-b27c-16099bbfadae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:58.769450 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8fda8-388e-4cf5-b27c-16099bbfadae None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:58.769941 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8fda8-388e-4cf5-b27c-16099bbfadae None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:58.770433 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8fda8-388e-4cf5-b27c-16099bbfadae None admin] PATH_INFO: `/v3/roles/service` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:58.798977 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b8fda8-388e-4cf5-b27c-16099bbfadae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:58.804553 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eae518d-fe30-4440-8a00-2cd47282dd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:58.805554 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae518d-fe30-4440-8a00-2cd47282dd9d None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:58.806016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae518d-fe30-4440-8a00-2cd47282dd9d None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:58.806474 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae518d-fe30-4440-8a00-2cd47282dd9d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:58.809505 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-a6b8fda8-388e-4cf5-b27c-16099bbfadae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:30:58.809505 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:58.809505 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:58.809505 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:58.809505 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:58.809505 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:58.809505 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:58.809505 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:58.809505 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:58.809505 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:58.809505 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:58.809505 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:58.809505 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:58.809505 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:58.809505 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:58.809505 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:58.810410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:58.810410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:58.810410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:58.810410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:58.810410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:58.810410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:58.810410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:58.810410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:58.810410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:58.810410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:58.810410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:58.810410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:58.810410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:58.810410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:58.810410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:58.810410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:58.811193 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:58.811193 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:58.811193 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:58.811193 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:58.811193 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:58.811193 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:58.811193 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:58.811193 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:58.811193 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:58.811193 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:58.811193 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:30:58.811193 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:30:58.812387 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 251/485] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 12:30:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:30:58.818056 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b8fda8-388e-4cf5-b27c-16099bbfadae None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:58.849189 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eae518d-fe30-4440-8a00-2cd47282dd9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:58.861157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-4eae518d-fe30-4440-8a00-2cd47282dd9d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:30:58.861157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:58.861157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:58.861157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:58.861157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:58.861157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:58.861157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:58.861157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:58.861157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:58.861157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:58.861157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:58.861157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:58.861157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:58.861157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:58.861157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:58.861157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:58.862042 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:58.862042 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:58.862042 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:58.862042 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:58.862042 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:58.862042 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:58.862042 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:58.862042 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:58.862042 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:58.862042 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:58.862042 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:58.862042 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:58.862042 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:58.862042 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:58.862042 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:58.862042 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:58.862816 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:58.862816 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:58.862816 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:58.862816 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:58.862816 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:58.862816 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:58.862816 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:58.862816 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:58.862816 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:58.862816 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:58.862816 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:30:58.862816 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:30:58.866957 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 235/486] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 12:30:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:30:58.868467 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f1c18-c2c5-4ebd-95ea-6083a910d66d None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:58.869098 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f1c18-c2c5-4ebd-95ea-6083a910d66d None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:58.870775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f1c18-c2c5-4ebd-95ea-6083a910d66d None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:58.873555 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 236/487] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:58.878448 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00f1c18-c2c5-4ebd-95ea-6083a910d66d None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:58.996461 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00f1c18-c2c5-4ebd-95ea-6083a910d66d None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:59.018506 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b8fda8-388e-4cf5-b27c-16099bbfadae None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:59.055352 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:59.062330 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c19939-86ec-4e3a-a488-301ad332eade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:59.062953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c19939-86ec-4e3a-a488-301ad332eade None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:59.063230 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c19939-86ec-4e3a-a488-301ad332eade None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:59.063531 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c19939-86ec-4e3a-a488-301ad332eade None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:59.064243 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c19939-86ec-4e3a-a488-301ad332eade None admin] RBAC: Adding query filter params (name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:59.064545 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c19939-86ec-4e3a-a488-301ad332eade None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:59.066177 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c19939-86ec-4e3a-a488-301ad332eade None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:59.069321 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1499be34-e756-4752-9805-5e84ede2f223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:59.072261 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1499be34-e756-4752-9805-5e84ede2f223 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:59.072261 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1499be34-e756-4752-9805-5e84ede2f223 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:59.072261 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1499be34-e756-4752-9805-5e84ede2f223 None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:59.072261 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1499be34-e756-4752-9805-5e84ede2f223 None admin] RBAC: Adding query filter params (name=service) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:59.072261 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1499be34-e756-4752-9805-5e84ede2f223 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:59.073706 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1499be34-e756-4752-9805-5e84ede2f223 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:59.080350 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 237/488] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 12:30:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:59.081732 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61549d28-544d-4194-be08-9fae301f632c None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:59.082026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61549d28-544d-4194-be08-9fae301f632c None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:59.082343 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61549d28-544d-4194-be08-9fae301f632c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:59.084129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 252/489] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 12:30:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:30:59.085153 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1499be34-e756-4752-9805-5e84ede2f223 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:59.102380 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-61549d28-544d-4194-be08-9fae301f632c None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:59.103730 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-61549d28-544d-4194-be08-9fae301f632c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:59.134041 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:59.179373 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 238/490] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:30:59.180214 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61549d28-544d-4194-be08-9fae301f632c None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:59.222635 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1499be34-e756-4752-9805-5e84ede2f223 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:59.296259 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc1766-89f4-4e11-b6e6-f4b92030e72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:59.296905 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc1766-89f4-4e11-b6e6-f4b92030e72b None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:59.297186 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc1766-89f4-4e11-b6e6-f4b92030e72b None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:59.297526 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc1766-89f4-4e11-b6e6-f4b92030e72b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:59.320607 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdc1766-89f4-4e11-b6e6-f4b92030e72b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:59.321979 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdc1766-89f4-4e11-b6e6-f4b92030e72b None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:59.354512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-7bdc1766-89f4-4e11-b6e6-f4b92030e72b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:30:59.354512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:59.354512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:59.354512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:59.354512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:59.354512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:59.354512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:59.354512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:59.354512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:59.354512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:59.354512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:59.354512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:59.354512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:30:59.354512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:30:59.354512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:30:59.354512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:30:59.355512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:59.355512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:59.355512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:59.355512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:59.355512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:59.355512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:59.355512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:59.355512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:59.355512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:59.355512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:59.355512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:59.355512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:59.355512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:59.355512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:59.355512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:59.355512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:59.355512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:30:59.356341 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:30:59.356341 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:30:59.356341 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:30:59.356341 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 253/491] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:30:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:30:59.369403 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc1766-89f4-4e11-b6e6-f4b92030e72b None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:59.372223 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61549d28-544d-4194-be08-9fae301f632c None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:59.416030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b5c56a-f7a9-4f74-8e8e-33c14220e5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:59.416716 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b5c56a-f7a9-4f74-8e8e-33c14220e5f9 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:59.416996 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b5c56a-f7a9-4f74-8e8e-33c14220e5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:59.417266 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b5c56a-f7a9-4f74-8e8e-33c14220e5f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:59.437859 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b5c56a-f7a9-4f74-8e8e-33c14220e5f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:59.439507 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b5c56a-f7a9-4f74-8e8e-33c14220e5f9 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:59.453558 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:59.458835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-c6b5c56a-f7a9-4f74-8e8e-33c14220e5f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:30:59.458835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:59.458835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:59.458835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:59.458835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:59.458835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:59.458835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:59.458835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:59.458835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:59.458835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:59.458835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:59.458835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:59.458835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:30:59.458835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:30:59.458835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:30:59.458835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:30:59.459844 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:59.459844 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:59.459844 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:59.459844 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:59.459844 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:59.459844 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:59.459844 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:59.459844 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:59.459844 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:59.459844 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:59.459844 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:59.459844 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:59.459844 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:59.459844 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:59.459844 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:59.459844 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:59.459844 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:30:59.460668 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:30:59.460668 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:30:59.460668 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:30:59.460668 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 239/492] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:30:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:30:59.470962 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b5c56a-f7a9-4f74-8e8e-33c14220e5f9 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:59.499613 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:59.528609 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc1766-89f4-4e11-b6e6-f4b92030e72b None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:59.565764 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba8b74e-56a0-48d2-94ca-d7917f188559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:59.566373 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8b74e-56a0-48d2-94ca-d7917f188559 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:59.566651 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8b74e-56a0-48d2-94ca-d7917f188559 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:59.566922 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8b74e-56a0-48d2-94ca-d7917f188559 None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:59.567712 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba8b74e-56a0-48d2-94ca-d7917f188559 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:59.568918 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba8b74e-56a0-48d2-94ca-d7917f188559 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:59.570294 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba8b74e-56a0-48d2-94ca-d7917f188559 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:59.582999 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 254/493] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:59.584124 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a4ec0-6f67-4a0b-a4f4-9d1c7495bc6a None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:59.584379 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a4ec0-6f67-4a0b-a4f4-9d1c7495bc6a None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:59.584834 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a4ec0-6f67-4a0b-a4f4-9d1c7495bc6a None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:59.586627 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 255/494] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:59.587287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a4ec0-6f67-4a0b-a4f4-9d1c7495bc6a None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:59.609046 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b5c56a-f7a9-4f74-8e8e-33c14220e5f9 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:59.645146 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde08753-e5af-40aa-b515-c22ad50c80aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:59.645762 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde08753-e5af-40aa-b515-c22ad50c80aa None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:59.646061 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde08753-e5af-40aa-b515-c22ad50c80aa None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:59.646343 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde08753-e5af-40aa-b515-c22ad50c80aa None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:59.647130 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde08753-e5af-40aa-b515-c22ad50c80aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:30:59.647477 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde08753-e5af-40aa-b515-c22ad50c80aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:59.648842 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde08753-e5af-40aa-b515-c22ad50c80aa None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:59.660464 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 240/495] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:30:59.665936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde08753-e5af-40aa-b515-c22ad50c80aa None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:59.695768 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:59.699750 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a4ec0-6f67-4a0b-a4f4-9d1c7495bc6a None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:59.736180 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e83aa1c-da77-46a4-b720-ee18c2f5869c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:59.736805 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83aa1c-da77-46a4-b720-ee18c2f5869c None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:59.737081 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83aa1c-da77-46a4-b720-ee18c2f5869c None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:59.737346 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83aa1c-da77-46a4-b720-ee18c2f5869c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:59.745872 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:59.750899 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e83aa1c-da77-46a4-b720-ee18c2f5869c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:30:59.756894 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-5e83aa1c-da77-46a4-b720-ee18c2f5869c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:30:59.756894 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:59.756894 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:59.756894 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:59.756894 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:59.756894 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:59.756894 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:59.756894 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:59.756894 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:59.756894 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:59.756894 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:59.756894 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:59.756894 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:30:59.756894 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:30:59.756894 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:30:59.756894 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:30:59.757790 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:30:59.757790 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:30:59.757790 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:30:59.757790 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:30:59.757790 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:59.757790 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:59.757790 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:59.757790 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:59.757790 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:59.757790 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:59.757790 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:59.757790 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:59.757790 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:59.757790 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:59.757790 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:59.757790 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:59.758561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:59.758561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:59.758561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:59.758561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:59.758561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:30:59.758561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:30:59.758561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:30:59.758561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:30:59.758561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:30:59.758561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:30:59.758561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:30:59.758561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:30:59.759607 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 256/496] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 12:30:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:30:59.760831 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3099a64b-f727-491c-b87e-05c32fcf89e3 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:59.761120 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3099a64b-f727-491c-b87e-05c32fcf89e3 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:59.761399 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3099a64b-f727-491c-b87e-05c32fcf89e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:59.768976 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde08753-e5af-40aa-b515-c22ad50c80aa None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:59.779328 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-3099a64b-f727-491c-b87e-05c32fcf89e3 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:30:59.780543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-3099a64b-f727-491c-b87e-05c32fcf89e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:30:59.803659 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108a619b-f20f-4a52-99c1-caa5273c2469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:59.804288 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a619b-f20f-4a52-99c1-caa5273c2469 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:59.804561 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a619b-f20f-4a52-99c1-caa5273c2469 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:59.804825 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a619b-f20f-4a52-99c1-caa5273c2469 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:59.812537 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108a619b-f20f-4a52-99c1-caa5273c2469 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:59.813986 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108a619b-f20f-4a52-99c1-caa5273c2469 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:30:59.827157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-108a619b-f20f-4a52-99c1-caa5273c2469 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 03 12:30:59.827157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:30:59.827157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:30:59.827157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:30:59.827157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:30:59.827157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:30:59.827157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:30:59.827157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:30:59.827157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:30:59.827157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:30:59.827157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:30:59.827157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:30:59.827157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 12:30:59.827157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 12:30:59.827157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 12:30:59.827157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 12:30:59.828131 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:30:59.828131 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:30:59.828131 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 12:30:59.828131 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:30:59.828131 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 12:30:59.828131 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:30:59.828131 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:30:59.828131 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:30:59.828131 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:30:59.828131 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:30:59.828131 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:30:59.828131 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:30:59.828131 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:30:59.828131 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:30:59.828131 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:30:59.828131 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:30:59.828131 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:30:59.828131 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:30:59.829030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:30:59.829030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:30:59.829030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 12:30:59.829030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 12:30:59.829030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 12:30:59.829030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 12:30:59.829030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 12:30:59.829030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 12:30:59.829030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 03 12:30:59.829030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:30:59.829030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 241/497] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:30:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:30:59.829030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108a619b-f20f-4a52-99c1-caa5273c2469 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:59.854830 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 257/498] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:30:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:30:59.856041 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0d300-aa83-4dcd-b6ec-eb8ecc8deaff None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:59.856330 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0d300-aa83-4dcd-b6ec-eb8ecc8deaff None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:59.856607 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0d300-aa83-4dcd-b6ec-eb8ecc8deaff None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:59.857986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 258/499] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:30:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:30:59.860056 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f0d300-aa83-4dcd-b6ec-eb8ecc8deaff None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:30:59.945570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108a619b-f20f-4a52-99c1-caa5273c2469 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:59.962668 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:59.973107 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f0d300-aa83-4dcd-b6ec-eb8ecc8deaff None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:30:59.983978 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be02f915-abf9-454b-a9bf-4af76dbcde00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:30:59.984617 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be02f915-abf9-454b-a9bf-4af76dbcde00 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:30:59.984896 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be02f915-abf9-454b-a9bf-4af76dbcde00 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:30:59.985171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be02f915-abf9-454b-a9bf-4af76dbcde00 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:30:59.988101 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:30:59.995220 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be02f915-abf9-454b-a9bf-4af76dbcde00 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:30:59.996774 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be02f915-abf9-454b-a9bf-4af76dbcde00 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:00.008752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-be02f915-abf9-454b-a9bf-4af76dbcde00 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 03 12:31:00.008752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:00.008752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:00.008752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:00.008752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:00.008752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:00.008752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:00.008752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:00.008752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:00.008752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:00.008752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:00.008752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:00.008752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 12:31:00.008752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 12:31:00.008752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 12:31:00.008752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 12:31:00.009853 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:00.009853 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:00.009853 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 12:31:00.009853 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:00.009853 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 12:31:00.009853 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:00.009853 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:00.009853 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:00.009853 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:00.009853 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:00.009853 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:00.009853 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:00.009853 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:00.009853 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:00.009853 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:00.009853 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:00.009853 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:00.009853 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:00.011147 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:00.011147 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:00.011147 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 12:31:00.011147 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 12:31:00.011147 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 12:31:00.011147 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 12:31:00.011147 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 12:31:00.011147 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 12:31:00.011147 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 03 12:31:00.011147 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:00.011147 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 242/500] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Oct 3 12:30:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:00.011147 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be02f915-abf9-454b-a9bf-4af76dbcde00 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:00.017434 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e929b39c-0d7c-4631-ad88-aee2c62ef17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:00.018053 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e929b39c-0d7c-4631-ad88-aee2c62ef17a None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:00.018333 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e929b39c-0d7c-4631-ad88-aee2c62ef17a None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:00.018604 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e929b39c-0d7c-4631-ad88-aee2c62ef17a None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:00.019266 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e929b39c-0d7c-4631-ad88-aee2c62ef17a None admin] RBAC: Adding query filter params (name=service) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:00.019596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e929b39c-0d7c-4631-ad88-aee2c62ef17a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:00.020950 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e929b39c-0d7c-4631-ad88-aee2c62ef17a None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:00.034565 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 259/501] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 12:30:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:00.034565 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd08482-1ac5-43f2-845a-ce055cc04bb8 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:00.034565 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd08482-1ac5-43f2-845a-ce055cc04bb8 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:00.034565 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd08482-1ac5-43f2-845a-ce055cc04bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:00.051684 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-6dd08482-1ac5-43f2-845a-ce055cc04bb8 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:00.053597 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-6dd08482-1ac5-43f2-845a-ce055cc04bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:00.119951 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be02f915-abf9-454b-a9bf-4af76dbcde00 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:00.130039 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 260/502] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:31:00.130917 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd08482-1ac5-43f2-845a-ce055cc04bb8 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:00.159400 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811bb689-bffe-4a24-847e-9d355ce20275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:00.160012 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bb689-bffe-4a24-847e-9d355ce20275 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:00.160432 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bb689-bffe-4a24-847e-9d355ce20275 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:00.160653 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bb689-bffe-4a24-847e-9d355ce20275 None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:00.161385 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811bb689-bffe-4a24-847e-9d355ce20275 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:00.162500 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811bb689-bffe-4a24-847e-9d355ce20275 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:00.164154 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811bb689-bffe-4a24-847e-9d355ce20275 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:00.201594 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 243/503] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Oct 3 12:31:00 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:00.202426 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811bb689-bffe-4a24-847e-9d355ce20275 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:00.223467 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:00.246736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd08482-1ac5-43f2-845a-ce055cc04bb8 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:00.267885 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:00.283868 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b618f4-7dbb-47cf-9e98-31658260d352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:00.284476 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b618f4-7dbb-47cf-9e98-31658260d352 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:00.284749 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b618f4-7dbb-47cf-9e98-31658260d352 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:00.285017 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b618f4-7dbb-47cf-9e98-31658260d352 None admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:00.285735 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b618f4-7dbb-47cf-9e98-31658260d352 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:00.286038 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b618f4-7dbb-47cf-9e98-31658260d352 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:00.287490 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b618f4-7dbb-47cf-9e98-31658260d352 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:00.316150 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811bb689-bffe-4a24-847e-9d355ce20275 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:00.329031 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 261/504] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Oct 3 12:31:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:00.329031 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b618f4-7dbb-47cf-9e98-31658260d352 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:00.357100 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be986a7-c56e-4ce2-a512-5e45e35a27a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:00.357750 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be986a7-c56e-4ce2-a512-5e45e35a27a2 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:00.358032 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be986a7-c56e-4ce2-a512-5e45e35a27a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:00.358295 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be986a7-c56e-4ce2-a512-5e45e35a27a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:00.369291 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be986a7-c56e-4ce2-a512-5e45e35a27a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:00.370792 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be986a7-c56e-4ce2-a512-5e45e35a27a2 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:00.387987 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-3be986a7-c56e-4ce2-a512-5e45e35a27a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:00.387987 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:00.387987 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:00.387987 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:00.387987 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:00.387987 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:00.387987 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:00.387987 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:00.387987 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:00.387987 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:00.387987 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:00.387987 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:00.387987 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:00.387987 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:00.387987 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:00.387987 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:00.388949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:00.388949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:00.388949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:00.388949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:00.388949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:00.388949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:00.388949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:00.388949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:00.388949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:00.388949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:00.388949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:00.388949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:00.388949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:00.388949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:00.388949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:00.388949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:00.388949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:00.389783 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:00.389783 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:00.389783 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:00.391930 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 244/505] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:00.393223 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be986a7-c56e-4ce2-a512-5e45e35a27a2 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:00.437632 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b618f4-7dbb-47cf-9e98-31658260d352 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:00.471118 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a684c6e-86bf-491f-a9b3-e580c8885d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:00.471783 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a684c6e-86bf-491f-a9b3-e580c8885d9d None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:00.472070 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a684c6e-86bf-491f-a9b3-e580c8885d9d None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:00.472345 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a684c6e-86bf-491f-a9b3-e580c8885d9d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:00.480023 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:00.482871 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a684c6e-86bf-491f-a9b3-e580c8885d9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:31:00.487705 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-8a684c6e-86bf-491f-a9b3-e580c8885d9d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:31:00.487705 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:00.487705 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:00.487705 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:00.487705 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:00.487705 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:00.487705 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:00.487705 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:00.487705 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:00.487705 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:00.487705 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:00.487705 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:00.487705 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:31:00.487705 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:31:00.487705 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:31:00.487705 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:31:00.494502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:31:00.494502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:31:00.494502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:31:00.494502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:31:00.494502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:00.494502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:00.494502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:00.494502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:00.494502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:00.494502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:00.494502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:00.494502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:00.494502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:00.494502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:00.494502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:00.494502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:00.495339 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:00.495339 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:00.495339 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:00.495339 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:00.495339 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:31:00.495339 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:31:00.495339 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:31:00.495339 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:31:00.495339 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:31:00.495339 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:31:00.495339 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:31:00.495339 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:00.495339 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 262/506] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 12:31:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:00.495339 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a684c6e-86bf-491f-a9b3-e580c8885d9d None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:00.496146 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be986a7-c56e-4ce2-a512-5e45e35a27a2 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:00.501905 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:00.529659 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534e243c-c8e3-4490-a5ac-f217f3a7ede0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:00.530262 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e243c-c8e3-4490-a5ac-f217f3a7ede0 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:00.530536 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e243c-c8e3-4490-a5ac-f217f3a7ede0 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:00.530802 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e243c-c8e3-4490-a5ac-f217f3a7ede0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:00.589750 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a684c6e-86bf-491f-a9b3-e580c8885d9d None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:00.597277 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534e243c-c8e3-4490-a5ac-f217f3a7ede0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:00.598731 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534e243c-c8e3-4490-a5ac-f217f3a7ede0 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:00.623600 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21756be-d4f8-4d36-8ef2-4fd99953d081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:00.624190 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21756be-d4f8-4d36-8ef2-4fd99953d081 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:00.624470 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21756be-d4f8-4d36-8ef2-4fd99953d081 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:00.624739 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21756be-d4f8-4d36-8ef2-4fd99953d081 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:00.688357 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:00.690957 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21756be-d4f8-4d36-8ef2-4fd99953d081 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:00.692447 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21756be-d4f8-4d36-8ef2-4fd99953d081 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:00.696656 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 245/507] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:31:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 12:31:00.697318 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534e243c-c8e3-4490-a5ac-f217f3a7ede0 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:00.707167 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:00.797222 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 263/508] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:31:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 12:31:00.797222 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21756be-d4f8-4d36-8ef2-4fd99953d081 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:00.814628 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534e243c-c8e3-4490-a5ac-f217f3a7ede0 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:00.851744 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771dc3c7-8850-48c5-a270-cd3c99a2757a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:00.852635 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771dc3c7-8850-48c5-a270-cd3c99a2757a None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:00.853056 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771dc3c7-8850-48c5-a270-cd3c99a2757a None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:00.853478 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771dc3c7-8850-48c5-a270-cd3c99a2757a None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:00.854451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771dc3c7-8850-48c5-a270-cd3c99a2757a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:00.854913 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771dc3c7-8850-48c5-a270-cd3c99a2757a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:00.857105 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771dc3c7-8850-48c5-a270-cd3c99a2757a None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:00.873549 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 246/509] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:00.874853 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771dc3c7-8850-48c5-a270-cd3c99a2757a None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:00.897922 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:00.900809 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21756be-d4f8-4d36-8ef2-4fd99953d081 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:00.936248 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212be4e1-e348-4fe6-a8c1-9f77ed13eee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:00.936877 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212be4e1-e348-4fe6-a8c1-9f77ed13eee6 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:00.937162 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212be4e1-e348-4fe6-a8c1-9f77ed13eee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:00.937441 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212be4e1-e348-4fe6-a8c1-9f77ed13eee6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:00.943735 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:00.947832 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212be4e1-e348-4fe6-a8c1-9f77ed13eee6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:00.949306 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212be4e1-e348-4fe6-a8c1-9f77ed13eee6 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:00.964024 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-212be4e1-e348-4fe6-a8c1-9f77ed13eee6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:00.964024 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:00.964024 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:00.964024 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:00.964024 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:00.964024 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:00.964024 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:00.964024 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:00.964024 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:00.964024 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:00.964024 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:00.964024 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:00.964024 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:00.964024 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:00.964024 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:00.964024 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:00.970117 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:00.970117 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:00.970117 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:00.970117 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:00.970117 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:00.970117 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:00.970117 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:00.970117 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:00.970117 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:00.970117 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:00.970117 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:00.970117 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:00.970117 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:00.970117 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:00.970117 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:00.970117 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:00.970117 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:00.971073 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:00.971073 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:00.971073 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:00.971073 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 264/510] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:00.971073 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212be4e1-e348-4fe6-a8c1-9f77ed13eee6 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:00.983452 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771dc3c7-8850-48c5-a270-cd3c99a2757a None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:01.017524 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd94d2-36c0-4d7b-afbf-1b2395896697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:01.018135 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd94d2-36c0-4d7b-afbf-1b2395896697 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:01.018418 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd94d2-36c0-4d7b-afbf-1b2395896697 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:01.018687 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd94d2-36c0-4d7b-afbf-1b2395896697 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:01.019364 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdd94d2-36c0-4d7b-afbf-1b2395896697 None admin] RBAC: Adding query filter params (name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:01.019670 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdd94d2-36c0-4d7b-afbf-1b2395896697 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:01.021545 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdd94d2-36c0-4d7b-afbf-1b2395896697 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:01.029957 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 247/511] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 12:31:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:01.030752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd94d2-36c0-4d7b-afbf-1b2395896697 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:01.071081 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212be4e1-e348-4fe6-a8c1-9f77ed13eee6 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:01.104499 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e69dc-7b26-4df3-b7b9-0794d0ae7a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:01.105122 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e69dc-7b26-4df3-b7b9-0794d0ae7a13 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:01.105397 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e69dc-7b26-4df3-b7b9-0794d0ae7a13 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:01.105667 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e69dc-7b26-4df3-b7b9-0794d0ae7a13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:01.116162 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086e69dc-7b26-4df3-b7b9-0794d0ae7a13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:01.117598 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086e69dc-7b26-4df3-b7b9-0794d0ae7a13 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:01.132514 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-086e69dc-7b26-4df3-b7b9-0794d0ae7a13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:01.132514 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:01.132514 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:01.132514 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:01.132514 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:01.132514 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:01.132514 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:01.132514 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:01.132514 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:01.132514 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:01.132514 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:01.132514 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:01.132514 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:01.132514 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:01.132514 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:01.132514 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:01.133812 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:01.133812 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:01.133812 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:01.133812 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:01.133812 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:01.133812 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:01.133812 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:01.133812 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:01.133812 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:01.133812 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:01.133812 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:01.133812 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:01.133812 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:01.133812 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:01.133812 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:01.133812 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:01.133812 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:01.134659 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:01.134659 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:01.134659 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:01.134659 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 265/512] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:01.134879 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd94d2-36c0-4d7b-afbf-1b2395896697 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:01.135226 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e69dc-7b26-4df3-b7b9-0794d0ae7a13 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:01.139096 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:01.163649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:01.171271 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0e875-fb3a-4a0c-b281-bdba66c31c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:01.171924 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0e875-fb3a-4a0c-b281-bdba66c31c33 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:01.172202 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0e875-fb3a-4a0c-b281-bdba66c31c33 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:01.172472 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0e875-fb3a-4a0c-b281-bdba66c31c33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:01.183221 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d0e875-fb3a-4a0c-b281-bdba66c31c33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:01.184783 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d0e875-fb3a-4a0c-b281-bdba66c31c33 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:01.199419 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-b0d0e875-fb3a-4a0c-b281-bdba66c31c33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:01.199419 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:01.199419 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:01.199419 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:01.199419 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:01.199419 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:01.199419 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:01.199419 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:01.199419 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:01.199419 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:01.199419 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:01.199419 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:01.199419 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:01.199419 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:01.199419 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:01.199419 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:01.200347 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:01.200347 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:01.200347 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:01.200347 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:01.200347 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:01.200347 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:01.200347 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:01.200347 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:01.200347 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:01.200347 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:01.200347 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:01.200347 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:01.200347 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:01.200347 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:01.200347 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:01.200347 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:01.200347 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:01.201179 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:01.201179 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:01.201179 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:01.201179 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 248/513] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:01.201399 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0e875-fb3a-4a0c-b281-bdba66c31c33 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:01.240706 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e69dc-7b26-4df3-b7b9-0794d0ae7a13 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:01.274276 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29712f39-ac38-4cb2-b1ec-09649f384243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:01.275056 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29712f39-ac38-4cb2-b1ec-09649f384243 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:01.275364 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29712f39-ac38-4cb2-b1ec-09649f384243 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:01.275642 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29712f39-ac38-4cb2-b1ec-09649f384243 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:01.283073 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29712f39-ac38-4cb2-b1ec-09649f384243 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:01.284530 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29712f39-ac38-4cb2-b1ec-09649f384243 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:01.296638 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-29712f39-ac38-4cb2-b1ec-09649f384243 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 03 12:31:01.296638 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:01.296638 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:01.296638 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:01.296638 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:01.296638 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:01.296638 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:01.296638 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:01.296638 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:01.296638 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:01.296638 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:01.296638 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:01.296638 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 12:31:01.296638 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 12:31:01.296638 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 12:31:01.296638 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 12:31:01.297600 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:01.297600 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:01.297600 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 12:31:01.297600 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:01.297600 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 12:31:01.297600 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:01.297600 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:01.297600 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:01.297600 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:01.297600 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:01.297600 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:01.297600 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:01.297600 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:01.297600 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:01.297600 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:01.297600 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:01.297600 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:01.297600 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:01.304035 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:01.304035 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:01.304035 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 12:31:01.304035 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 12:31:01.304035 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 12:31:01.304035 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 12:31:01.304035 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 12:31:01.304035 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 12:31:01.304035 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 03 12:31:01.304035 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:01.304035 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 266/514] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:31:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:01.304035 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29712f39-ac38-4cb2-b1ec-09649f384243 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:01.304786 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0e875-fb3a-4a0c-b281-bdba66c31c33 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:01.338143 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e132433-b687-4519-9530-3b617c7fca31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:01.338745 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e132433-b687-4519-9530-3b617c7fca31 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:01.339019 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e132433-b687-4519-9530-3b617c7fca31 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:01.339292 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e132433-b687-4519-9530-3b617c7fca31 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:01.339960 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e132433-b687-4519-9530-3b617c7fca31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:01.340255 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e132433-b687-4519-9530-3b617c7fca31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:01.341869 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e132433-b687-4519-9530-3b617c7fca31 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:01.353715 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 249/515] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:01.354505 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e132433-b687-4519-9530-3b617c7fca31 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:01.368269 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:01.395873 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:01.405275 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29712f39-ac38-4cb2-b1ec-09649f384243 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:01.438158 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec38c3-4c81-40c4-9396-874fb87424bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:01.438785 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec38c3-4c81-40c4-9396-874fb87424bd None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:01.439065 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec38c3-4c81-40c4-9396-874fb87424bd None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:01.439367 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec38c3-4c81-40c4-9396-874fb87424bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:01.450162 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ec38c3-4c81-40c4-9396-874fb87424bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:01.451623 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ec38c3-4c81-40c4-9396-874fb87424bd None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:01.455423 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e132433-b687-4519-9530-3b617c7fca31 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:01.465994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-98ec38c3-4c81-40c4-9396-874fb87424bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:01.465994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:01.465994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:01.465994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:01.465994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:01.465994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:01.465994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:01.465994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:01.465994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:01.465994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:01.465994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:01.465994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:01.465994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:01.465994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:01.465994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:01.465994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:01.467168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:01.467168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:01.467168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:01.467168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:01.467168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:01.467168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:01.467168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:01.467168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:01.467168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:01.467168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:01.467168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:01.467168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:01.467168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:01.467168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:01.467168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:01.467168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:01.467168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:01.468066 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:01.468066 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:01.468066 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:01.468066 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 267/516] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:01.468419 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec38c3-4c81-40c4-9396-874fb87424bd None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:01.490373 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72bdf2e-95be-4727-8f95-39494fe53ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:01.490981 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72bdf2e-95be-4727-8f95-39494fe53ee8 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:01.491259 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72bdf2e-95be-4727-8f95-39494fe53ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:01.491575 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72bdf2e-95be-4727-8f95-39494fe53ee8 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:01.492211 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72bdf2e-95be-4727-8f95-39494fe53ee8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:01.492515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72bdf2e-95be-4727-8f95-39494fe53ee8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:01.493880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72bdf2e-95be-4727-8f95-39494fe53ee8 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:01.506275 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 250/517] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:01.507060 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72bdf2e-95be-4727-8f95-39494fe53ee8 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:01.569051 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec38c3-4c81-40c4-9396-874fb87424bd None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:01.585665 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:01.601211 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:01.603507 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a905054c-f126-4017-91bf-0319dc3783d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:01.604121 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905054c-f126-4017-91bf-0319dc3783d7 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:01.604247 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72bdf2e-95be-4727-8f95-39494fe53ee8 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:01.604408 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905054c-f126-4017-91bf-0319dc3783d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:01.604680 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905054c-f126-4017-91bf-0319dc3783d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:01.605316 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a905054c-f126-4017-91bf-0319dc3783d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:01.605630 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a905054c-f126-4017-91bf-0319dc3783d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:01.606986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a905054c-f126-4017-91bf-0319dc3783d7 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:01.618736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 268/518] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:01.619756 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a905054c-f126-4017-91bf-0319dc3783d7 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:01.639563 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1100ef2-927a-4b1f-b5d0-15932725f053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:01.640176 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1100ef2-927a-4b1f-b5d0-15932725f053 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:01.640457 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1100ef2-927a-4b1f-b5d0-15932725f053 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:01.640733 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1100ef2-927a-4b1f-b5d0-15932725f053 None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:01.641454 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1100ef2-927a-4b1f-b5d0-15932725f053 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:01.641757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1100ef2-927a-4b1f-b5d0-15932725f053 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:01.643184 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1100ef2-927a-4b1f-b5d0-15932725f053 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:01.660969 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 251/519] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Oct 3 12:31:01 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:01.661736 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1100ef2-927a-4b1f-b5d0-15932725f053 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:01.718733 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a905054c-f126-4017-91bf-0319dc3783d7 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:01.752268 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c453d0-3664-442f-a4d1-527cb1ffe95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:01.752868 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c453d0-3664-442f-a4d1-527cb1ffe95c None admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:01.753144 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c453d0-3664-442f-a4d1-527cb1ffe95c None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:01.753411 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c453d0-3664-442f-a4d1-527cb1ffe95c None admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:01.754037 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c453d0-3664-442f-a4d1-527cb1ffe95c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:01.756014 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c453d0-3664-442f-a4d1-527cb1ffe95c None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:01.760332 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1100ef2-927a-4b1f-b5d0-15932725f053 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:01.794089 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0afa80-ae6d-4011-a673-339c2a056f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:01.794693 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0afa80-ae6d-4011-a673-339c2a056f30 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:01.794976 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0afa80-ae6d-4011-a673-339c2a056f30 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:01.795252 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0afa80-ae6d-4011-a673-339c2a056f30 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:01.795920 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0afa80-ae6d-4011-a673-339c2a056f30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:01.796224 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0afa80-ae6d-4011-a673-339c2a056f30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:01.797604 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0afa80-ae6d-4011-a673-339c2a056f30 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:01.809152 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 252/520] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:01.809933 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0afa80-ae6d-4011-a673-339c2a056f30 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:01.829358 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:01.843644 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 269/521] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Oct 3 12:31:01 2022] POST /identity/v3/users => generated 247 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:31:01.844461 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c453d0-3664-442f-a4d1-527cb1ffe95c None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:01.889334 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:01.934122 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0afa80-ae6d-4011-a673-339c2a056f30 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:01.971731 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c453d0-3664-442f-a4d1-527cb1ffe95c None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:01.973498 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9a96c-cd40-46c3-95a2-78af4a63b4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:01.974321 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9a96c-cd40-46c3-95a2-78af4a63b4e6 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:01.974761 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9a96c-cd40-46c3-95a2-78af4a63b4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:01.975159 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9a96c-cd40-46c3-95a2-78af4a63b4e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:01.982699 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d9a96c-cd40-46c3-95a2-78af4a63b4e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:01.984370 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d9a96c-cd40-46c3-95a2-78af4a63b4e6 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:01.998225 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-f2d9a96c-cd40-46c3-95a2-78af4a63b4e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:01.998225 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:01.998225 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:01.998225 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:01.998225 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:01.998225 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:01.998225 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:01.998225 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:01.998225 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:01.998225 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:01.998225 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:01.998225 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:01.998225 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 12:31:01.998225 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 12:31:01.998225 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 12:31:01.999341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 12:31:01.999341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:01.999341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:01.999341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:01.999341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:01.999341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:01.999341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:01.999341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:01.999341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:01.999341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:01.999341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:01.999341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:01.999341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:01.999341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:01.999341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:01.999341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:01.999341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:02.000206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 12:31:02.000206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 12:31:02.000206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 12:31:02.000206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 12:31:02.000206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 12:31:02.000206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 12:31:02.000206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:02.000206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:02.000206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 253/522] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:31:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:02.000688 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9a96c-cd40-46c3-95a2-78af4a63b4e6 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:02.020826 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df264dd-c72e-49a1-974d-2cc3566ad4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:02.022051 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df264dd-c72e-49a1-974d-2cc3566ad4e0 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:02.022376 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df264dd-c72e-49a1-974d-2cc3566ad4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:02.022651 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df264dd-c72e-49a1-974d-2cc3566ad4e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:02.030721 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df264dd-c72e-49a1-974d-2cc3566ad4e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:02.032226 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df264dd-c72e-49a1-974d-2cc3566ad4e0 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:02.045002 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-1df264dd-c72e-49a1-974d-2cc3566ad4e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:02.045002 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:02.045002 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:02.045002 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:02.045002 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:02.045002 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:02.045002 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:02.045002 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:02.045002 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:02.045002 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:02.045002 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:02.045002 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:02.045002 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 12:31:02.045002 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 12:31:02.045002 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 12:31:02.045870 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 12:31:02.045870 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:02.045870 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:02.045870 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:02.045870 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:02.045870 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:02.045870 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:02.045870 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:02.045870 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:02.045870 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:02.045870 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:02.045870 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:02.045870 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:02.045870 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:02.045870 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:02.045870 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:02.045870 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:02.046741 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 12:31:02.046741 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 12:31:02.046741 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 12:31:02.046741 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 12:31:02.046741 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 12:31:02.046741 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 12:31:02.046741 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:02.046741 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:02.046741 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 270/523] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:31:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:02.047236 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df264dd-c72e-49a1-974d-2cc3566ad4e0 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:02.092235 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:02.116012 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9a96c-cd40-46c3-95a2-78af4a63b4e6 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:02.155031 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:02.156290 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa2d6a-b419-4a41-897a-3451a28baae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:02.156964 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa2d6a-b419-4a41-897a-3451a28baae3 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:02.157373 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa2d6a-b419-4a41-897a-3451a28baae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:02.157714 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa2d6a-b419-4a41-897a-3451a28baae3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:02.164958 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df264dd-c72e-49a1-974d-2cc3566ad4e0 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:02.200438 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ccea4-4756-465b-af5e-8843e1134bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:02.201085 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ccea4-4756-465b-af5e-8843e1134bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:02.201432 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ccea4-4756-465b-af5e-8843e1134bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:02.201786 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ccea4-4756-465b-af5e-8843e1134bc7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:02.209173 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29ccea4-4756-465b-af5e-8843e1134bc7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:02.210716 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29ccea4-4756-465b-af5e-8843e1134bc7 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:02.223480 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-b29ccea4-4756-465b-af5e-8843e1134bc7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 03 12:31:02.223480 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:02.223480 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:02.223480 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:02.223480 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:02.223480 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:02.223480 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:02.223480 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:02.223480 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:02.223480 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:02.223480 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:02.223480 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:02.223480 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 12:31:02.223480 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 12:31:02.223480 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 12:31:02.223480 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 12:31:02.225785 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:02.225785 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:02.225785 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 12:31:02.225785 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:02.225785 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 12:31:02.225785 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:02.225785 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:02.225785 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:02.225785 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:02.225785 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:02.225785 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:02.225785 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:02.225785 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:02.225785 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:02.225785 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:02.225785 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:02.225785 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:02.225785 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:02.227019 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:02.227019 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:02.227019 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 12:31:02.227019 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 12:31:02.227019 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 12:31:02.227019 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 12:31:02.227019 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 12:31:02.227019 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 12:31:02.227019 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 03 12:31:02.227019 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:02.231146 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aa2d6a-b419-4a41-897a-3451a28baae3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:02.232397 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 271/524] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 12:31:02 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:02.232643 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aa2d6a-b419-4a41-897a-3451a28baae3 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:02.233694 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ccea4-4756-465b-af5e-8843e1134bc7 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:02.319973 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:02.343382 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 254/525] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:31:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 12:31:02.343819 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ccea4-4756-465b-af5e-8843e1134bc7 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:02.344228 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa2d6a-b419-4a41-897a-3451a28baae3 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:02.383678 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042026f-064c-49ac-9368-68142ae1726a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:02.383918 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042026f-064c-49ac-9368-68142ae1726a None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:02.384155 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042026f-064c-49ac-9368-68142ae1726a None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:02.384314 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042026f-064c-49ac-9368-68142ae1726a None admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:02.384992 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5042026f-064c-49ac-9368-68142ae1726a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:02.385295 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5042026f-064c-49ac-9368-68142ae1726a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:02.386716 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5042026f-064c-49ac-9368-68142ae1726a None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:02.406278 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 272/526] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 12:31:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:02.407100 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042026f-064c-49ac-9368-68142ae1726a None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:02.422447 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:02.455957 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa2d6a-b419-4a41-897a-3451a28baae3 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:02.491587 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ae514-fb9c-4868-a039-1a2470621f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:02.492196 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ae514-fb9c-4868-a039-1a2470621f85 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:02.492480 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ae514-fb9c-4868-a039-1a2470621f85 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:02.492735 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ae514-fb9c-4868-a039-1a2470621f85 None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:02.493415 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010ae514-fb9c-4868-a039-1a2470621f85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:02.493713 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010ae514-fb9c-4868-a039-1a2470621f85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:02.495132 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010ae514-fb9c-4868-a039-1a2470621f85 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:02.512554 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042026f-064c-49ac-9368-68142ae1726a None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:02.513511 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 255/527] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 12:31:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:02.519029 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ae514-fb9c-4868-a039-1a2470621f85 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:02.550510 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f3219-1702-43ae-9e47-92230c043356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:02.551326 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f3219-1702-43ae-9e47-92230c043356 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:02.551713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f3219-1702-43ae-9e47-92230c043356 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:02.552067 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f3219-1702-43ae-9e47-92230c043356 None admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:02.553437 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10f3219-1702-43ae-9e47-92230c043356 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:02.553895 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10f3219-1702-43ae-9e47-92230c043356 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:02.555459 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10f3219-1702-43ae-9e47-92230c043356 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:02.561043 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:02.577275 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 273/528] 10.222.0.33 () {56 vars in 1139 bytes} [Mon Oct 3 12:31:02 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:02.577275 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f3219-1702-43ae-9e47-92230c043356 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:02.630452 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ae514-fb9c-4868-a039-1a2470621f85 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:02.650222 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:02.668228 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabcc36e-0b67-47ab-85d8-14d26cde4d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:02.668864 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabcc36e-0b67-47ab-85d8-14d26cde4d1a None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:02.669144 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabcc36e-0b67-47ab-85d8-14d26cde4d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:02.669417 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabcc36e-0b67-47ab-85d8-14d26cde4d1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:02.679913 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabcc36e-0b67-47ab-85d8-14d26cde4d1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:02.681367 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabcc36e-0b67-47ab-85d8-14d26cde4d1a None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:02.683014 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f3219-1702-43ae-9e47-92230c043356 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:02.696427 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-eabcc36e-0b67-47ab-85d8-14d26cde4d1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:02.696427 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:02.696427 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:02.696427 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:02.696427 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:02.696427 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:02.696427 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:02.696427 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:02.696427 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:02.696427 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:02.696427 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:02.696427 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:02.696427 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:02.696427 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:02.696427 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:02.696427 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:02.697515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:02.697515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:02.697515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:02.697515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:02.697515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:02.697515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:02.697515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:02.697515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:02.697515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:02.697515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:02.697515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:02.697515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:02.697515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:02.697515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:02.697515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:02.697515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:02.697515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:02.698544 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:02.698544 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:02.698544 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:02.699913 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 256/529] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:02.700881 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabcc36e-0b67-47ab-85d8-14d26cde4d1a None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:02.724891 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c76f5b-8ff5-42d9-a70e-629298f9c37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:02.725179 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c76f5b-8ff5-42d9-a70e-629298f9c37c None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:02.725403 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c76f5b-8ff5-42d9-a70e-629298f9c37c None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:02.725679 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c76f5b-8ff5-42d9-a70e-629298f9c37c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:02.726394 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c76f5b-8ff5-42d9-a70e-629298f9c37c None admin] RBAC: Adding query filter params (role.id=ff6b2d4413644db39a2bd8576afc6ccb, scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7, user.id=524e9184eee1486cb071cad5c67050c9) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:02.726701 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c76f5b-8ff5-42d9-a70e-629298f9c37c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:02.728139 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c76f5b-8ff5-42d9-a70e-629298f9c37c None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:02.733895 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 274/530] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:31:02 2022] GET /identity/v3/role_assignments?user.id=524e9184eee1486cb071cad5c67050c9&scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7&role.id=ff6b2d4413644db39a2bd8576afc6ccb => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:02.734811 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c76f5b-8ff5-42d9-a70e-629298f9c37c None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:02.831366 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:02.843921 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabcc36e-0b67-47ab-85d8-14d26cde4d1a None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:02.880555 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c76f5b-8ff5-42d9-a70e-629298f9c37c None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:02.885412 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e846eb6-1b54-4702-83cd-792a494d5cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:02.886281 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e846eb6-1b54-4702-83cd-792a494d5cda None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:02.886774 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e846eb6-1b54-4702-83cd-792a494d5cda None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:02.887224 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e846eb6-1b54-4702-83cd-792a494d5cda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:02.888204 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e846eb6-1b54-4702-83cd-792a494d5cda None admin] RBAC: Adding query filter params (role.id=ff6b2d4413644db39a2bd8576afc6ccb, scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7, user.id=990f5837994f4a088c656c9b33e9a0dc) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:02.888717 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e846eb6-1b54-4702-83cd-792a494d5cda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:02.890398 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e846eb6-1b54-4702-83cd-792a494d5cda None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:02.898100 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 257/531] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:31:02 2022] GET /identity/v3/role_assignments?user.id=990f5837994f4a088c656c9b33e9a0dc&scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7&role.id=ff6b2d4413644db39a2bd8576afc6ccb => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:02.899440 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e846eb6-1b54-4702-83cd-792a494d5cda None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:02.921606 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480d9aab-9817-41fe-a170-3b145471e353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:02.922221 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d9aab-9817-41fe-a170-3b145471e353 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:02.922503 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d9aab-9817-41fe-a170-3b145471e353 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:02.922778 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d9aab-9817-41fe-a170-3b145471e353 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:02.925976 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:03.016259 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480d9aab-9817-41fe-a170-3b145471e353 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:03.017736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480d9aab-9817-41fe-a170-3b145471e353 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:03.048852 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e846eb6-1b54-4702-83cd-792a494d5cda None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:03.088476 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f8ca8b-8e02-498a-9423-b1614f499ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:03.089476 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f8ca8b-8e02-498a-9423-b1614f499ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:03.089968 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f8ca8b-8e02-498a-9423-b1614f499ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:03.090439 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f8ca8b-8e02-498a-9423-b1614f499ca4 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:03.091277 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f8ca8b-8e02-498a-9423-b1614f499ca4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:03.092099 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f8ca8b-8e02-498a-9423-b1614f499ca4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:03.093769 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f8ca8b-8e02-498a-9423-b1614f499ca4 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:03.108705 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 258/532] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:03.114547 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f8ca8b-8e02-498a-9423-b1614f499ca4 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:03.133946 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:03.149862 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 275/533] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:31:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 12:31:03.157796 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480d9aab-9817-41fe-a170-3b145471e353 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:03.208466 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:03.306815 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f8ca8b-8e02-498a-9423-b1614f499ca4 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:03.335952 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480d9aab-9817-41fe-a170-3b145471e353 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:03.376916 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce483c-ec08-438b-85ba-128e4cb9ed97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:03.378002 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce483c-ec08-438b-85ba-128e4cb9ed97 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:03.378434 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce483c-ec08-438b-85ba-128e4cb9ed97 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:03.378883 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce483c-ec08-438b-85ba-128e4cb9ed97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:03.392574 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ce483c-ec08-438b-85ba-128e4cb9ed97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:03.394347 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ce483c-ec08-438b-85ba-128e4cb9ed97 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:03.411713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-85ce483c-ec08-438b-85ba-128e4cb9ed97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:03.411713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:03.411713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:03.411713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:03.411713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:03.411713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:03.411713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:03.411713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:03.411713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:03.411713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:03.411713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:03.411713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:03.411713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:03.411713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:03.411713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:03.411713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:03.413475 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:03.413475 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:03.413475 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:03.413475 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:03.413475 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:03.413475 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:03.413475 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:03.413475 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:03.413475 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:03.413475 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:03.413475 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:03.413475 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:03.413475 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:03.413475 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:03.413475 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:03.413475 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:03.413475 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:03.414427 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:03.414427 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:03.414427 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:03.415555 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e153dd-74e0-4349-80ec-ebfe83db46ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:03.416338 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e153dd-74e0-4349-80ec-ebfe83db46ee None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:03.416529 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 276/534] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:31:03.423400 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e153dd-74e0-4349-80ec-ebfe83db46ee None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:03.423400 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e153dd-74e0-4349-80ec-ebfe83db46ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:03.423603 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce483c-ec08-438b-85ba-128e4cb9ed97 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:03.434026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e153dd-74e0-4349-80ec-ebfe83db46ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:03.440240 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e153dd-74e0-4349-80ec-ebfe83db46ee None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:03.456868 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-e4e153dd-74e0-4349-80ec-ebfe83db46ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:03.456868 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:03.456868 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:03.456868 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:03.456868 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:03.456868 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:03.456868 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:03.456868 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:03.456868 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:03.456868 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:03.456868 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:03.456868 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:03.456868 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 12:31:03.456868 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 12:31:03.456868 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 12:31:03.457907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 12:31:03.457907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:03.457907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:03.457907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:03.457907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:03.457907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:03.457907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:03.457907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:03.457907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:03.457907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:03.457907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:03.457907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:03.457907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:03.457907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:03.457907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:03.457907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:03.457907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:03.459597 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 12:31:03.459597 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 12:31:03.459597 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 12:31:03.459597 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 12:31:03.459597 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 12:31:03.459597 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 12:31:03.459597 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:03.459597 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:03.462206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 259/535] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:31:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:03.472276 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e153dd-74e0-4349-80ec-ebfe83db46ee None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:03.546347 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:03.563848 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce483c-ec08-438b-85ba-128e4cb9ed97 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:03.608290 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd77246-9aeb-47c0-941c-6755633e0ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:03.612182 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd77246-9aeb-47c0-941c-6755633e0ab0 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:03.612612 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd77246-9aeb-47c0-941c-6755633e0ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:03.613025 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd77246-9aeb-47c0-941c-6755633e0ab0 None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:03.613803 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd77246-9aeb-47c0-941c-6755633e0ab0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:03.614270 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd77246-9aeb-47c0-941c-6755633e0ab0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:03.618163 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd77246-9aeb-47c0-941c-6755633e0ab0 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:03.641691 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 277/536] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:03.647862 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd77246-9aeb-47c0-941c-6755633e0ab0 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:03.665723 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:03.718949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e153dd-74e0-4349-80ec-ebfe83db46ee None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:03.789279 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2676b2-2f19-4a11-b4ad-7d24b67eab3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:03.790132 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2676b2-2f19-4a11-b4ad-7d24b67eab3c None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:03.792252 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2676b2-2f19-4a11-b4ad-7d24b67eab3c None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:03.792671 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2676b2-2f19-4a11-b4ad-7d24b67eab3c None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:03.793503 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2676b2-2f19-4a11-b4ad-7d24b67eab3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:03.795182 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2676b2-2f19-4a11-b4ad-7d24b67eab3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:03.796774 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2676b2-2f19-4a11-b4ad-7d24b67eab3c None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:03.814520 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd77246-9aeb-47c0-941c-6755633e0ab0 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:03.828207 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 260/537] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 12:31:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:03.853557 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2676b2-2f19-4a11-b4ad-7d24b67eab3c None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:03.884256 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3089391-3e09-4363-8d09-c28e03fe7686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:03.885212 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3089391-3e09-4363-8d09-c28e03fe7686 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:03.887889 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3089391-3e09-4363-8d09-c28e03fe7686 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:03.888306 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3089391-3e09-4363-8d09-c28e03fe7686 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:03.902762 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3089391-3e09-4363-8d09-c28e03fe7686 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:03.905787 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3089391-3e09-4363-8d09-c28e03fe7686 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:03.929283 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-e3089391-3e09-4363-8d09-c28e03fe7686 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:03.929283 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:03.929283 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:03.929283 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:03.929283 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:03.929283 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:03.929283 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:03.929283 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:03.929283 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:03.929283 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:03.929283 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:03.929283 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:03.929283 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 12:31:03.929283 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 12:31:03.929283 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 12:31:03.930331 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 12:31:03.930331 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:03.930331 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:03.930331 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:03.930331 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:03.930331 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:03.930331 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:03.930331 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:03.930331 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:03.930331 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:03.930331 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:03.930331 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:03.930331 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:03.930331 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:03.930331 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:03.930331 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:03.930331 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:03.931376 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 12:31:03.931376 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 12:31:03.931376 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 12:31:03.931376 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 12:31:03.931376 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 12:31:03.931376 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 12:31:03.931376 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:03.931376 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:03.940297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 278/538] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:31:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:03.946212 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3089391-3e09-4363-8d09-c28e03fe7686 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:03.964303 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:04.021356 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2676b2-2f19-4a11-b4ad-7d24b67eab3c None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:04.061312 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b920a96-3658-4913-a7a5-dc397d150cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:04.062154 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b920a96-3658-4913-a7a5-dc397d150cf0 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:04.062589 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b920a96-3658-4913-a7a5-dc397d150cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:04.062997 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b920a96-3658-4913-a7a5-dc397d150cf0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:04.063895 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b920a96-3658-4913-a7a5-dc397d150cf0 None admin] RBAC: Adding query filter params (role.id=ff6b2d4413644db39a2bd8576afc6ccb, scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7, user.id=b697e4847b13496d98a543675f5df1c4) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:04.064358 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b920a96-3658-4913-a7a5-dc397d150cf0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:04.065999 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b920a96-3658-4913-a7a5-dc397d150cf0 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:04.073097 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 261/539] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:31:03 2022] GET /identity/v3/role_assignments?user.id=b697e4847b13496d98a543675f5df1c4&scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7&role.id=ff6b2d4413644db39a2bd8576afc6ccb => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:04.075653 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:04.141827 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3089391-3e09-4363-8d09-c28e03fe7686 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:04.152326 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911abbd7-a8f6-4949-84fe-b255e63c1901 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:04.152941 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911abbd7-a8f6-4949-84fe-b255e63c1901 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:04.153826 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911abbd7-a8f6-4949-84fe-b255e63c1901 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:04.156416 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 262/540] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:04.163224 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f45a858-4c7d-4cb7-906f-bf1321fc7640 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:04.163558 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f45a858-4c7d-4cb7-906f-bf1321fc7640 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:04.163801 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f45a858-4c7d-4cb7-906f-bf1321fc7640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:04.185754 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-9f45a858-4c7d-4cb7-906f-bf1321fc7640 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:04.186982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-9f45a858-4c7d-4cb7-906f-bf1321fc7640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:04.203973 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f33310-c63f-430a-acd1-078a278cb03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:04.204676 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f33310-c63f-430a-acd1-078a278cb03e None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:04.204944 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f33310-c63f-430a-acd1-078a278cb03e None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:04.205219 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f33310-c63f-430a-acd1-078a278cb03e None admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:04.205974 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f33310-c63f-430a-acd1-078a278cb03e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:04.206287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f33310-c63f-430a-acd1-078a278cb03e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:04.214596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f33310-c63f-430a-acd1-078a278cb03e None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:04.243627 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 279/541] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 12:31:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:31:04.258082 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f33310-c63f-430a-acd1-078a278cb03e None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:04.323916 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 263/542] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 12:31:04.367364 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:04.403578 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f33310-c63f-430a-acd1-078a278cb03e None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:04.447184 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17bbd31-4aa5-4e28-bcda-b7640999745d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:04.447827 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17bbd31-4aa5-4e28-bcda-b7640999745d None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:04.448106 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17bbd31-4aa5-4e28-bcda-b7640999745d None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:04.448372 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17bbd31-4aa5-4e28-bcda-b7640999745d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:04.449095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17bbd31-4aa5-4e28-bcda-b7640999745d None admin] RBAC: Adding query filter params (role.id=ff6b2d4413644db39a2bd8576afc6ccb, scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7, user.id=eb327a8a18ca414ab8b4111a4d13aad8) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:04.449401 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17bbd31-4aa5-4e28-bcda-b7640999745d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:04.450816 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17bbd31-4aa5-4e28-bcda-b7640999745d None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:04.456595 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 280/543] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:31:04 2022] GET /identity/v3/role_assignments?user.id=eb327a8a18ca414ab8b4111a4d13aad8&scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7&role.id=ff6b2d4413644db39a2bd8576afc6ccb => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:04.820052 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f45a858-4c7d-4cb7-906f-bf1321fc7640 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:04.958147 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f45a858-4c7d-4cb7-906f-bf1321fc7640 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:05.000294 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:05.012529 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0509ef-c802-48e6-9b33-d7c6a8dbe13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:05.013621 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0509ef-c802-48e6-9b33-d7c6a8dbe13a None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:05.032766 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0509ef-c802-48e6-9b33-d7c6a8dbe13a None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:05.032766 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0509ef-c802-48e6-9b33-d7c6a8dbe13a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:05.032766 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0509ef-c802-48e6-9b33-d7c6a8dbe13a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:31:05.032766 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-2b0509ef-c802-48e6-9b33-d7c6a8dbe13a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:31:05.032766 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:05.032766 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:05.032766 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:05.032766 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:05.032766 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:05.032766 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:05.033360 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:05.033360 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:05.033360 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:05.033360 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:05.033360 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:05.033360 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:31:05.033360 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:31:05.033360 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:31:05.033360 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:31:05.033360 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:31:05.033360 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:31:05.033360 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:31:05.033360 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:31:05.033360 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:05.033360 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:05.033360 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:05.033360 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:05.034193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:05.034193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:05.034193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:05.034193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:05.034193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:05.034193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:05.034193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:05.034193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:05.034193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:05.034193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:05.034193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:05.034193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:05.034193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:31:05.034193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:31:05.034193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:31:05.034193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:31:05.034193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:31:05.035017 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:31:05.035017 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:31:05.035017 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:05.035017 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 264/544] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 12:31:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:05.037585 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17bbd31-4aa5-4e28-bcda-b7640999745d None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:05.177124 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17bbd31-4aa5-4e28-bcda-b7640999745d None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:05.206532 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:05.218100 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd3da73-87ec-40f2-ae8c-cf101959e7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:05.218721 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3da73-87ec-40f2-ae8c-cf101959e7b7 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:05.219002 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3da73-87ec-40f2-ae8c-cf101959e7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:05.219274 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3da73-87ec-40f2-ae8c-cf101959e7b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:05.219959 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd3da73-87ec-40f2-ae8c-cf101959e7b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:05.220260 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd3da73-87ec-40f2-ae8c-cf101959e7b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:05.221664 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd3da73-87ec-40f2-ae8c-cf101959e7b7 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:05.231259 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 281/545] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 12:31:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:05.238378 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0509ef-c802-48e6-9b33-d7c6a8dbe13a None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:05.409340 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0509ef-c802-48e6-9b33-d7c6a8dbe13a None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:05.429329 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9721b-a995-46ac-83d5-6e3c9eb5d3bf None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:05.429636 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9721b-a995-46ac-83d5-6e3c9eb5d3bf None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:05.429829 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9721b-a995-46ac-83d5-6e3c9eb5d3bf None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:05.431536 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 282/546] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:05.438467 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3230948a-b78c-407e-88e3-22bc9f7b3bc7 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:05.438767 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3230948a-b78c-407e-88e3-22bc9f7b3bc7 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:05.439021 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3230948a-b78c-407e-88e3-22bc9f7b3bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:05.475177 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-3230948a-b78c-407e-88e3-22bc9f7b3bc7 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:05.476429 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-3230948a-b78c-407e-88e3-22bc9f7b3bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:05.481743 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921563b2-d2f7-4862-b28f-b467da8f19fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:05.482389 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921563b2-d2f7-4862-b28f-b467da8f19fd None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:05.482669 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921563b2-d2f7-4862-b28f-b467da8f19fd None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:05.482938 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921563b2-d2f7-4862-b28f-b467da8f19fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:05.502875 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921563b2-d2f7-4862-b28f-b467da8f19fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:05.504376 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921563b2-d2f7-4862-b28f-b467da8f19fd None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:05.530157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-921563b2-d2f7-4862-b28f-b467da8f19fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:05.530157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:05.530157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:05.530157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:05.530157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:05.530157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:05.530157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:05.530157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:05.530157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:05.530157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:05.530157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:05.530157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:05.530157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:05.530157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:05.530157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:05.530157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:05.531159 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:05.531159 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:05.531159 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:05.531159 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:05.531159 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:05.531159 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:05.531159 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:05.531159 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:05.531159 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:05.531159 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:05.531159 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:05.531159 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:05.531159 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:05.531159 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:05.531159 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:05.531159 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:05.531159 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:05.532524 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:05.532524 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:05.532524 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:05.532524 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 265/547] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:05.537886 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921563b2-d2f7-4862-b28f-b467da8f19fd None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:05.597236 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:05.603643 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 283/548] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:31:05.605254 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fefae3b-46ba-473d-9c3c-d43603182f1c None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:05.605730 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fefae3b-46ba-473d-9c3c-d43603182f1c None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:05.606190 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fefae3b-46ba-473d-9c3c-d43603182f1c None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:05.608152 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 284/549] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:05.613991 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208b175-7e6d-42d2-8cca-b75acb68a1c1 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:05.614647 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208b175-7e6d-42d2-8cca-b75acb68a1c1 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:05.615651 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208b175-7e6d-42d2-8cca-b75acb68a1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:05.635026 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-e208b175-7e6d-42d2-8cca-b75acb68a1c1 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:05.636520 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-e208b175-7e6d-42d2-8cca-b75acb68a1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:05.664414 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921563b2-d2f7-4862-b28f-b467da8f19fd None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:05.700515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ffd5d-c254-4cba-986c-99eb0295e415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:05.701138 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ffd5d-c254-4cba-986c-99eb0295e415 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:05.701492 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ffd5d-c254-4cba-986c-99eb0295e415 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:05.701847 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ffd5d-c254-4cba-986c-99eb0295e415 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:05.702644 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ffd5d-c254-4cba-986c-99eb0295e415 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:05.703042 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ffd5d-c254-4cba-986c-99eb0295e415 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:05.704634 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ffd5d-c254-4cba-986c-99eb0295e415 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:05.709093 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 285/550] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:31:05.718900 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 266/551] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:05.723861 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e208b175-7e6d-42d2-8cca-b75acb68a1c1 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:05.791655 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:05.860857 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e208b175-7e6d-42d2-8cca-b75acb68a1c1 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:05.909010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da4412-42b6-4f42-bd80-7d6705e39287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:05.909645 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da4412-42b6-4f42-bd80-7d6705e39287 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:05.909927 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da4412-42b6-4f42-bd80-7d6705e39287 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:05.910199 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da4412-42b6-4f42-bd80-7d6705e39287 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:05.922088 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76da4412-42b6-4f42-bd80-7d6705e39287 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:05.923552 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76da4412-42b6-4f42-bd80-7d6705e39287 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:05.939615 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-76da4412-42b6-4f42-bd80-7d6705e39287 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 03 12:31:05.939615 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:05.939615 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:05.939615 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:05.939615 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:05.939615 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:05.939615 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:05.939615 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:05.939615 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:05.939615 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:05.939615 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:05.939615 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:05.939615 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 12:31:05.939615 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 12:31:05.939615 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 12:31:05.939615 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 12:31:05.940561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:05.940561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:05.940561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 12:31:05.940561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:05.940561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 12:31:05.940561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:05.940561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:05.940561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:05.940561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:05.940561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:05.940561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:05.940561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:05.940561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:05.940561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:05.940561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:05.940561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:05.940561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:05.940561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:05.941547 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:05.941547 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:05.941547 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 12:31:05.941547 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 12:31:05.941547 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 12:31:05.941547 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 12:31:05.941547 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 12:31:05.941547 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 12:31:05.941547 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 03 12:31:05.941547 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:05.941547 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 286/552] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 12:31:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:05.947180 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ffd5d-c254-4cba-986c-99eb0295e415 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:05.962522 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da4412-42b6-4f42-bd80-7d6705e39287 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:06.063222 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da4412-42b6-4f42-bd80-7d6705e39287 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:06.077757 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:06.101520 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1990ee0-f77c-4aff-afc9-14e6a07a5240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:06.103596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1990ee0-f77c-4aff-afc9-14e6a07a5240 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:06.104143 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1990ee0-f77c-4aff-afc9-14e6a07a5240 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:06.105323 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1990ee0-f77c-4aff-afc9-14e6a07a5240 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:06.120373 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1990ee0-f77c-4aff-afc9-14e6a07a5240 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:31:06.125867 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-f1990ee0-f77c-4aff-afc9-14e6a07a5240 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:31:06.125867 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:06.125867 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:06.125867 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:06.125867 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:06.125867 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:06.125867 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:06.125867 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:06.125867 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:06.125867 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:06.125867 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:06.125867 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:06.125867 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:31:06.125867 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:31:06.125867 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:31:06.125867 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:31:06.126749 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:31:06.126749 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:31:06.126749 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:31:06.126749 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:31:06.126749 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:06.126749 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:06.126749 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:06.126749 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:06.126749 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:06.126749 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:06.126749 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:06.126749 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:06.126749 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:06.126749 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:06.126749 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:06.126749 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:06.127617 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:06.127617 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:06.127617 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:06.127617 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:06.127617 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:31:06.127617 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:31:06.127617 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:31:06.127617 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:31:06.127617 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:31:06.127617 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:31:06.127617 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:31:06.127617 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:06.127617 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 287/553] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 12:31:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:31:06.128297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1990ee0-f77c-4aff-afc9-14e6a07a5240 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:06.157935 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:06.163034 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ffd5d-c254-4cba-986c-99eb0295e415 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:06.215198 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e366f3-a02e-4c75-9dfb-1eb9ade532bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:06.215844 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e366f3-a02e-4c75-9dfb-1eb9ade532bd None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:06.216131 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e366f3-a02e-4c75-9dfb-1eb9ade532bd None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:06.216409 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e366f3-a02e-4c75-9dfb-1eb9ade532bd None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:06.217693 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e366f3-a02e-4c75-9dfb-1eb9ade532bd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:06.218006 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e366f3-a02e-4c75-9dfb-1eb9ade532bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:06.219814 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e366f3-a02e-4c75-9dfb-1eb9ade532bd None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:06.246188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 267/554] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Oct 3 12:31:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:06.249498 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e366f3-a02e-4c75-9dfb-1eb9ade532bd None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:06.266646 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1990ee0-f77c-4aff-afc9-14e6a07a5240 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:06.304868 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace90057-77b7-44f9-8feb-9d8006f2f37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:06.305899 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace90057-77b7-44f9-8feb-9d8006f2f37f None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:06.306380 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace90057-77b7-44f9-8feb-9d8006f2f37f None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:06.306881 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace90057-77b7-44f9-8feb-9d8006f2f37f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:06.317811 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace90057-77b7-44f9-8feb-9d8006f2f37f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:31:06.323986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-ace90057-77b7-44f9-8feb-9d8006f2f37f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:31:06.323986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:06.323986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:06.323986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:06.323986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:06.323986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:06.323986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:06.323986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:06.323986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:06.323986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:06.323986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:06.323986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:06.323986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:31:06.323986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:31:06.323986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:31:06.323986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:31:06.324888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:31:06.324888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:31:06.324888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:31:06.324888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:31:06.324888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:06.324888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:06.324888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:06.324888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:06.324888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:06.324888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:06.324888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:06.324888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:06.324888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:06.324888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:06.324888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:06.324888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:06.325692 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:06.325692 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:06.325692 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:06.325692 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:06.325692 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:31:06.325692 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:31:06.325692 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:31:06.325692 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:31:06.325692 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:31:06.325692 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:31:06.325692 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:31:06.325692 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:06.328212 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 288/555] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 12:31:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:06.329297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace90057-77b7-44f9-8feb-9d8006f2f37f None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:06.370903 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e366f3-a02e-4c75-9dfb-1eb9ade532bd None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:06.372878 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:06.408081 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6382ad2d-1a00-4861-90d0-41f184f305f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:06.409059 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382ad2d-1a00-4861-90d0-41f184f305f2 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:06.409532 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382ad2d-1a00-4861-90d0-41f184f305f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:06.410436 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382ad2d-1a00-4861-90d0-41f184f305f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:06.411295 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6382ad2d-1a00-4861-90d0-41f184f305f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:06.412100 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6382ad2d-1a00-4861-90d0-41f184f305f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:06.413646 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6382ad2d-1a00-4861-90d0-41f184f305f2 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:06.422733 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 268/556] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 12:31:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:06.423883 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6382ad2d-1a00-4861-90d0-41f184f305f2 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:06.447564 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:06.468459 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace90057-77b7-44f9-8feb-9d8006f2f37f None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:06.505941 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1103de2-0cf6-41a2-8627-f32d9acab902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:06.506905 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1103de2-0cf6-41a2-8627-f32d9acab902 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:06.507440 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1103de2-0cf6-41a2-8627-f32d9acab902 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:06.508167 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1103de2-0cf6-41a2-8627-f32d9acab902 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:06.563758 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6382ad2d-1a00-4861-90d0-41f184f305f2 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:06.598990 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1103de2-0cf6-41a2-8627-f32d9acab902 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:06.602090 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1103de2-0cf6-41a2-8627-f32d9acab902 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:06.612509 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573e17de-9280-4864-9cbc-d96e563b2496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:06.613127 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e17de-9280-4864-9cbc-d96e563b2496 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:06.613479 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e17de-9280-4864-9cbc-d96e563b2496 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:06.613763 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e17de-9280-4864-9cbc-d96e563b2496 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:06.614956 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573e17de-9280-4864-9cbc-d96e563b2496 None admin] RBAC: Adding query filter params (name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:06.615478 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573e17de-9280-4864-9cbc-d96e563b2496 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:06.617338 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573e17de-9280-4864-9cbc-d96e563b2496 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:06.628360 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 269/557] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 12:31:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:06.629896 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573e17de-9280-4864-9cbc-d96e563b2496 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:06.639991 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:06.718079 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 289/558] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:31:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 12:31:06.719336 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1103de2-0cf6-41a2-8627-f32d9acab902 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:06.730045 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:06.750996 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573e17de-9280-4864-9cbc-d96e563b2496 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:06.788350 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b5341-9c52-4423-b6ce-9fc5128ed404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:06.789040 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b5341-9c52-4423-b6ce-9fc5128ed404 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:06.789322 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b5341-9c52-4423-b6ce-9fc5128ed404 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:06.789598 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b5341-9c52-4423-b6ce-9fc5128ed404 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:06.799696 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023b5341-9c52-4423-b6ce-9fc5128ed404 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:06.801145 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023b5341-9c52-4423-b6ce-9fc5128ed404 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:06.821702 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-023b5341-9c52-4423-b6ce-9fc5128ed404 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:06.821702 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:06.821702 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:06.821702 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:06.821702 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:06.821702 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:06.821702 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:06.821702 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:06.821702 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:06.821702 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:06.821702 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:06.821702 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:06.821702 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:06.821702 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:06.821702 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:06.821702 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:06.822703 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:06.822703 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:06.822703 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:06.822703 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:06.822703 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:06.822703 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:06.822703 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:06.822703 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:06.822703 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:06.822703 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:06.822703 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:06.822703 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:06.822703 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:06.822703 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:06.822703 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:06.822703 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:06.822703 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:06.823626 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:06.823626 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:06.823626 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:06.823626 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 270/559] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:06.824210 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8f4c0-1ef4-49ae-b155-646f07fb8e83 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:06.824283 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8f4c0-1ef4-49ae-b155-646f07fb8e83 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:06.824502 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8f4c0-1ef4-49ae-b155-646f07fb8e83 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:06.829567 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 271/560] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:06.830326 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8f4c0-1ef4-49ae-b155-646f07fb8e83 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:06.845561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1103de2-0cf6-41a2-8627-f32d9acab902 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:06.884018 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:06.889390 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab74f27-5ec7-4b5c-8272-d02398236b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:06.890049 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab74f27-5ec7-4b5c-8272-d02398236b08 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:06.890322 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab74f27-5ec7-4b5c-8272-d02398236b08 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:06.890592 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab74f27-5ec7-4b5c-8272-d02398236b08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:06.903567 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab74f27-5ec7-4b5c-8272-d02398236b08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:06.905106 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab74f27-5ec7-4b5c-8272-d02398236b08 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:06.924070 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-2ab74f27-5ec7-4b5c-8272-d02398236b08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:06.924070 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:06.924070 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:06.924070 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:06.924070 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:06.924070 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:06.924070 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:06.924070 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:06.924070 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:06.924070 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:06.924070 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:06.924070 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:06.924070 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:06.924070 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:06.924070 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:06.924070 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:06.925069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:06.925069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:06.925069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:06.925069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:06.925069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:06.925069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:06.925069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:06.925069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:06.925069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:06.925069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:06.925069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:06.925069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:06.925069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:06.925069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:06.925069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:06.925069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:06.925069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:06.925947 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:06.925947 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:06.925947 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:06.927366 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 290/561] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:06.929116 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab74f27-5ec7-4b5c-8272-d02398236b08 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:06.961078 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8f4c0-1ef4-49ae-b155-646f07fb8e83 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:07.004574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc124d3-a762-476b-800d-24cd3203a7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:07.005218 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc124d3-a762-476b-800d-24cd3203a7a4 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:07.006030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc124d3-a762-476b-800d-24cd3203a7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:07.006908 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc124d3-a762-476b-800d-24cd3203a7a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:07.009480 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:07.018534 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc124d3-a762-476b-800d-24cd3203a7a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:07.020082 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc124d3-a762-476b-800d-24cd3203a7a4 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:07.035745 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-1bc124d3-a762-476b-800d-24cd3203a7a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:07.035745 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:07.035745 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:07.035745 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:07.035745 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:07.035745 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:07.035745 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:07.035745 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:07.035745 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:07.035745 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:07.035745 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:07.035745 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:07.035745 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:07.035745 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:07.035745 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:07.035745 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:07.036985 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:07.036985 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:07.036985 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:07.036985 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:07.036985 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:07.036985 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:07.036985 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:07.036985 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:07.036985 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:07.036985 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:07.036985 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:07.036985 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:07.036985 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:07.036985 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:07.036985 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:07.036985 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:07.036985 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:07.038078 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:07.038078 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:07.038078 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:07.038078 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 272/562] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:07.038742 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1489841-6885-4f73-a231-6f9ea4fefbdb None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:07.039027 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1489841-6885-4f73-a231-6f9ea4fefbdb None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:07.039306 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1489841-6885-4f73-a231-6f9ea4fefbdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:07.045079 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab74f27-5ec7-4b5c-8272-d02398236b08 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:07.058891 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-d1489841-6885-4f73-a231-6f9ea4fefbdb None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:07.060180 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d1489841-6885-4f73-a231-6f9ea4fefbdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:07.079175 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb45c79-ab7e-4b08-9bb7-8a1f728bf02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:07.079881 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb45c79-ab7e-4b08-9bb7-8a1f728bf02a None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:07.080168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb45c79-ab7e-4b08-9bb7-8a1f728bf02a None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:07.080442 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb45c79-ab7e-4b08-9bb7-8a1f728bf02a None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:07.081078 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb45c79-ab7e-4b08-9bb7-8a1f728bf02a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:07.081380 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb45c79-ab7e-4b08-9bb7-8a1f728bf02a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:07.082761 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb45c79-ab7e-4b08-9bb7-8a1f728bf02a None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:07.094593 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 291/563] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:07.095413 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb45c79-ab7e-4b08-9bb7-8a1f728bf02a None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:07.133547 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 273/564] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:31:07.134987 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0174e4-63c2-4c78-adc3-09e7f3c9f16f None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:07.135286 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0174e4-63c2-4c78-adc3-09e7f3c9f16f None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:07.135589 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0174e4-63c2-4c78-adc3-09e7f3c9f16f None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:07.136818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 274/565] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:07.137511 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0174e4-63c2-4c78-adc3-09e7f3c9f16f None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:07.149655 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:07.215913 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb45c79-ab7e-4b08-9bb7-8a1f728bf02a None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:07.254133 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0174e4-63c2-4c78-adc3-09e7f3c9f16f None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:07.257353 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94c736-3bb2-4757-8b83-bab2dbf35c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:07.257911 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94c736-3bb2-4757-8b83-bab2dbf35c7c None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:07.258195 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94c736-3bb2-4757-8b83-bab2dbf35c7c None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:07.258531 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94c736-3bb2-4757-8b83-bab2dbf35c7c None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:07.259178 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f94c736-3bb2-4757-8b83-bab2dbf35c7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:07.259569 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f94c736-3bb2-4757-8b83-bab2dbf35c7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:07.260938 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f94c736-3bb2-4757-8b83-bab2dbf35c7c None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:07.274036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:07.277797 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 292/566] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:07.278515 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94c736-3bb2-4757-8b83-bab2dbf35c7c None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:07.293328 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9891795c-3a16-4077-8437-d8e38aea2ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:07.293939 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891795c-3a16-4077-8437-d8e38aea2ac1 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:07.294222 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891795c-3a16-4077-8437-d8e38aea2ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:07.294498 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891795c-3a16-4077-8437-d8e38aea2ac1 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:07.295140 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9891795c-3a16-4077-8437-d8e38aea2ac1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:07.295478 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9891795c-3a16-4077-8437-d8e38aea2ac1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:07.296895 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9891795c-3a16-4077-8437-d8e38aea2ac1 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:07.312762 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 275/567] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:07.312762 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390cef3-13d3-406f-888b-90c57db9c784 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:07.312762 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390cef3-13d3-406f-888b-90c57db9c784 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:07.312762 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390cef3-13d3-406f-888b-90c57db9c784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:07.328132 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-2390cef3-13d3-406f-888b-90c57db9c784 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:07.329329 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-2390cef3-13d3-406f-888b-90c57db9c784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:07.394955 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94c736-3bb2-4757-8b83-bab2dbf35c7c None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:07.398282 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:07.404249 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 276/568] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:31:07.405105 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390cef3-13d3-406f-888b-90c57db9c784 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:07.436530 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357e8c4-d856-4e2e-92d8-d7b46a553807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:07.437324 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357e8c4-d856-4e2e-92d8-d7b46a553807 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:07.437608 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357e8c4-d856-4e2e-92d8-d7b46a553807 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:07.437876 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357e8c4-d856-4e2e-92d8-d7b46a553807 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:07.446519 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9357e8c4-d856-4e2e-92d8-d7b46a553807 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:07.448006 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9357e8c4-d856-4e2e-92d8-d7b46a553807 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:07.459450 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-9357e8c4-d856-4e2e-92d8-d7b46a553807 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 03 12:31:07.459450 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:07.459450 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:07.459450 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:07.459450 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:07.459450 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:07.459450 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:07.459450 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:07.459450 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:07.459450 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:07.459450 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:07.459450 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:07.459450 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 12:31:07.459450 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 12:31:07.459450 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 12:31:07.459450 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 12:31:07.460388 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:07.460388 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:07.460388 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 12:31:07.460388 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:07.460388 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 12:31:07.460388 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:07.460388 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:07.460388 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:07.460388 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:07.460388 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:07.460388 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:07.460388 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:07.460388 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:07.460388 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:07.460388 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:07.460388 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:07.460388 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:07.460388 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:07.461311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:07.461311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:07.461311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 12:31:07.461311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 12:31:07.461311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 12:31:07.461311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 12:31:07.461311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 12:31:07.461311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 12:31:07.461311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 03 12:31:07.461311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:07.461311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 293/569] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:31:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:07.461891 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357e8c4-d856-4e2e-92d8-d7b46a553807 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:07.522875 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390cef3-13d3-406f-888b-90c57db9c784 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:07.533062 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:07.567061 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9583e6-18db-49f1-b062-2082dd54d7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:07.567726 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9583e6-18db-49f1-b062-2082dd54d7f4 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:07.568574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9583e6-18db-49f1-b062-2082dd54d7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:07.569133 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9583e6-18db-49f1-b062-2082dd54d7f4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:07.576001 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9583e6-18db-49f1-b062-2082dd54d7f4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:07.577350 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9583e6-18db-49f1-b062-2082dd54d7f4 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:07.579011 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357e8c4-d856-4e2e-92d8-d7b46a553807 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:07.588984 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-2d9583e6-18db-49f1-b062-2082dd54d7f4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 03 12:31:07.588984 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:07.588984 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:07.588984 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:07.588984 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:07.588984 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:07.588984 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:07.588984 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:07.588984 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:07.588984 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:07.588984 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:07.588984 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:07.588984 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 12:31:07.588984 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 12:31:07.588984 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 12:31:07.588984 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 12:31:07.589979 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:07.589979 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:07.589979 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 12:31:07.589979 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:07.589979 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 12:31:07.589979 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:07.589979 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:07.589979 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:07.589979 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:07.589979 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:07.589979 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:07.589979 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:07.589979 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:07.589979 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:07.589979 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:07.589979 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:07.589979 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:07.589979 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:07.590911 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:07.590911 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:07.590911 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 12:31:07.590911 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 12:31:07.590911 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 12:31:07.590911 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 12:31:07.590911 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 12:31:07.590911 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 12:31:07.590911 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 03 12:31:07.590911 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:07.590911 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 277/570] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Oct 3 12:31:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:07.590911 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9583e6-18db-49f1-b062-2082dd54d7f4 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:07.625949 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba924ce1-2afc-48d2-ac38-a07d9ed38526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:07.626874 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba924ce1-2afc-48d2-ac38-a07d9ed38526 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:07.627425 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba924ce1-2afc-48d2-ac38-a07d9ed38526 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:07.627913 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba924ce1-2afc-48d2-ac38-a07d9ed38526 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:07.635521 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba924ce1-2afc-48d2-ac38-a07d9ed38526 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:07.637061 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba924ce1-2afc-48d2-ac38-a07d9ed38526 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:07.648828 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-ba924ce1-2afc-48d2-ac38-a07d9ed38526 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:07.648828 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:07.648828 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:07.648828 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:07.648828 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:07.648828 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:07.648828 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:07.648828 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:07.648828 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:07.648828 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:07.648828 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:07.648828 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:07.648828 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 12:31:07.648828 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 12:31:07.648828 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 12:31:07.649934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 12:31:07.649934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:07.649934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:07.649934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:07.649934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:07.649934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:07.649934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:07.649934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:07.649934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:07.649934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:07.649934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:07.649934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:07.649934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:07.649934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:07.649934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:07.649934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:07.649934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:07.651082 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 12:31:07.651082 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 12:31:07.651082 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 12:31:07.651082 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 12:31:07.651082 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 12:31:07.651082 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 12:31:07.651082 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:07.651082 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:07.651082 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 294/571] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:31:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:07.651082 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba924ce1-2afc-48d2-ac38-a07d9ed38526 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:07.677036 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:07.700828 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9583e6-18db-49f1-b062-2082dd54d7f4 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:07.734460 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26d40b-20d5-4564-bf20-62f59a2fe21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:07.735081 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26d40b-20d5-4564-bf20-62f59a2fe21c None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:07.735382 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26d40b-20d5-4564-bf20-62f59a2fe21c None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:07.735651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26d40b-20d5-4564-bf20-62f59a2fe21c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:07.746137 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc26d40b-20d5-4564-bf20-62f59a2fe21c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:31:07.750784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-fc26d40b-20d5-4564-bf20-62f59a2fe21c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:31:07.750784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:07.750784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:07.750784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:07.750784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:07.750784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:07.750784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:07.750784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:07.750784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:07.750784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:07.750784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:07.750784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:07.750784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:31:07.750784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:31:07.750784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:31:07.750784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:31:07.751828 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:31:07.751828 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:31:07.751828 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:31:07.751828 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:31:07.751828 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:07.751828 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:07.751828 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:07.751828 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:07.751828 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:07.751828 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:07.751828 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:07.751828 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:07.751828 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:07.751828 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:07.751828 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:07.751828 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:07.752689 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:07.752689 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:07.752689 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:07.752689 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:07.752689 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:31:07.752689 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:31:07.752689 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:31:07.752689 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:31:07.752689 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:31:07.752689 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:31:07.752689 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:31:07.752689 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:07.753311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba924ce1-2afc-48d2-ac38-a07d9ed38526 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:07.753393 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 278/572] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 12:31:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:07.753393 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26d40b-20d5-4564-bf20-62f59a2fe21c None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:07.780792 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:07.786493 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2365cd3-b513-47ba-a29d-fa8003f528ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:07.787422 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2365cd3-b513-47ba-a29d-fa8003f528ad None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:07.787708 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2365cd3-b513-47ba-a29d-fa8003f528ad None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:07.787982 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2365cd3-b513-47ba-a29d-fa8003f528ad None admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:07.788789 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2365cd3-b513-47ba-a29d-fa8003f528ad None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:07.789085 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2365cd3-b513-47ba-a29d-fa8003f528ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:07.790524 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2365cd3-b513-47ba-a29d-fa8003f528ad None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:07.812357 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 295/573] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Oct 3 12:31:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:07.812357 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2365cd3-b513-47ba-a29d-fa8003f528ad None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:07.853836 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26d40b-20d5-4564-bf20-62f59a2fe21c None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:07.888456 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54929c1-e647-490f-b42e-71feca8e1a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:07.889062 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54929c1-e647-490f-b42e-71feca8e1a4d None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:07.889339 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54929c1-e647-490f-b42e-71feca8e1a4d None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:07.889610 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54929c1-e647-490f-b42e-71feca8e1a4d None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:07.890323 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54929c1-e647-490f-b42e-71feca8e1a4d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:07.890624 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54929c1-e647-490f-b42e-71feca8e1a4d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:07.892062 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54929c1-e647-490f-b42e-71feca8e1a4d None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:07.896787 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:07.909453 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 279/574] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Oct 3 12:31:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:07.911455 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54929c1-e647-490f-b42e-71feca8e1a4d None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:07.915032 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2365cd3-b513-47ba-a29d-fa8003f528ad None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:07.948818 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732b644-585c-4fc2-9794-4021741ca1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:07.949424 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732b644-585c-4fc2-9794-4021741ca1b4 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:07.949704 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732b644-585c-4fc2-9794-4021741ca1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:07.949977 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732b644-585c-4fc2-9794-4021741ca1b4 None admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:07.950656 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9732b644-585c-4fc2-9794-4021741ca1b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:07.950960 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9732b644-585c-4fc2-9794-4021741ca1b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:07.952426 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9732b644-585c-4fc2-9794-4021741ca1b4 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:07.964769 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 296/575] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 12:31:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:07.966308 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732b644-585c-4fc2-9794-4021741ca1b4 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:07.996765 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:08.012041 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54929c1-e647-490f-b42e-71feca8e1a4d None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:08.045938 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf1bd14-881e-4d68-b1a3-664db361cad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:08.046501 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1bd14-881e-4d68-b1a3-664db361cad2 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:08.046783 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1bd14-881e-4d68-b1a3-664db361cad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:08.047143 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1bd14-881e-4d68-b1a3-664db361cad2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:08.057620 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf1bd14-881e-4d68-b1a3-664db361cad2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:31:08.062451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-adf1bd14-881e-4d68-b1a3-664db361cad2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:31:08.062451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:08.062451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:08.062451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:08.062451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:08.062451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:08.062451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:08.062451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:08.062451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:08.062451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:08.062451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:08.062451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:08.062451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:31:08.062451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:31:08.062451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:31:08.062451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:31:08.063440 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:31:08.063440 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:31:08.063440 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:31:08.063440 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:31:08.063440 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:08.063440 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:08.063440 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:08.063440 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:08.063440 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:08.063440 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:08.063440 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:08.063440 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:08.063440 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:08.063440 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:08.063440 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:08.063440 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:08.064278 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:08.064278 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:08.064278 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:08.064278 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:08.064278 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:31:08.064278 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:31:08.064278 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:31:08.064278 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:31:08.064278 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:31:08.064278 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:31:08.064278 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:31:08.064278 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:08.064278 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 280/576] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 12:31:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:08.064963 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf1bd14-881e-4d68-b1a3-664db361cad2 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:08.070605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732b644-585c-4fc2-9794-4021741ca1b4 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:08.104377 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f805d95d-7560-440b-8247-c8be2c03bf94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:08.104985 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805d95d-7560-440b-8247-c8be2c03bf94 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:08.105269 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805d95d-7560-440b-8247-c8be2c03bf94 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:08.105542 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805d95d-7560-440b-8247-c8be2c03bf94 None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:08.106196 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f805d95d-7560-440b-8247-c8be2c03bf94 None admin] RBAC: Adding query filter params (name=service) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:08.106500 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f805d95d-7560-440b-8247-c8be2c03bf94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:08.107879 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f805d95d-7560-440b-8247-c8be2c03bf94 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:08.115639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 297/577] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 12:31:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:08.116416 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f805d95d-7560-440b-8247-c8be2c03bf94 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:08.120223 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:08.167240 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf1bd14-881e-4d68-b1a3-664db361cad2 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:08.201073 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9dd1ef-33e9-43d9-8535-a96afdbd6bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:08.201676 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9dd1ef-33e9-43d9-8535-a96afdbd6bcb None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:08.201956 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9dd1ef-33e9-43d9-8535-a96afdbd6bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:08.202223 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9dd1ef-33e9-43d9-8535-a96afdbd6bcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:08.212519 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9dd1ef-33e9-43d9-8535-a96afdbd6bcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:08.213689 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f805d95d-7560-440b-8247-c8be2c03bf94 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:08.213940 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9dd1ef-33e9-43d9-8535-a96afdbd6bcb None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:08.216501 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:08.230105 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-8e9dd1ef-33e9-43d9-8535-a96afdbd6bcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:08.230105 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:08.230105 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:08.230105 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:08.230105 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:08.230105 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:08.230105 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:08.230105 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:08.230105 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:08.230105 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:08.230105 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:08.230105 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:08.230105 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:08.230105 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:08.230105 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:08.230105 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:08.232148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:08.232148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:08.232148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:08.232148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:08.232148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:08.232148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:08.232148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:08.232148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:08.232148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:08.232148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:08.232148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:08.232148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:08.232148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:08.232148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:08.232148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:08.232148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:08.232148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:08.233277 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:08.233277 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:08.233277 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:08.233277 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 281/578] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:08.233277 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9dd1ef-33e9-43d9-8535-a96afdbd6bcb None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:08.250381 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5fe76d-a041-404c-ad17-3a347c713d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:08.251019 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5fe76d-a041-404c-ad17-3a347c713d49 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:08.251299 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5fe76d-a041-404c-ad17-3a347c713d49 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:08.251597 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5fe76d-a041-404c-ad17-3a347c713d49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:08.262616 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5fe76d-a041-404c-ad17-3a347c713d49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:08.264080 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5fe76d-a041-404c-ad17-3a347c713d49 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:08.278440 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-6b5fe76d-a041-404c-ad17-3a347c713d49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:08.278440 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:08.278440 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:08.278440 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:08.278440 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:08.278440 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:08.278440 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:08.278440 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:08.278440 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:08.278440 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:08.278440 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:08.278440 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:08.278440 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:08.278440 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:08.278440 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:08.278440 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:08.279852 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:08.279852 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:08.279852 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:08.279852 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:08.279852 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:08.279852 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:08.279852 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:08.279852 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:08.279852 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:08.279852 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:08.279852 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:08.279852 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:08.279852 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:08.279852 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:08.279852 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:08.279852 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:08.279852 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:08.280731 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:08.280731 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:08.280731 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:08.280731 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 298/579] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:08.280731 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5fe76d-a041-404c-ad17-3a347c713d49 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:08.338837 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9dd1ef-33e9-43d9-8535-a96afdbd6bcb None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:08.339344 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:08.372849 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e1300-ea84-495b-a09e-c251d8b4d035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:08.373447 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e1300-ea84-495b-a09e-c251d8b4d035 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:08.373729 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e1300-ea84-495b-a09e-c251d8b4d035 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:08.373996 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e1300-ea84-495b-a09e-c251d8b4d035 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:08.374704 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43e1300-ea84-495b-a09e-c251d8b4d035 None admin] RBAC: Adding query filter params (role.id=ff6b2d4413644db39a2bd8576afc6ccb, scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7, user.id=b111d72a34144073917de4f566a58233) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:08.375140 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43e1300-ea84-495b-a09e-c251d8b4d035 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:08.376920 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43e1300-ea84-495b-a09e-c251d8b4d035 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:08.383806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 282/580] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:31:08 2022] GET /identity/v3/role_assignments?user.id=b111d72a34144073917de4f566a58233&scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7&role.id=ff6b2d4413644db39a2bd8576afc6ccb => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:08.384582 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e1300-ea84-495b-a09e-c251d8b4d035 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:08.385247 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5fe76d-a041-404c-ad17-3a347c713d49 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:08.423895 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec245c-277b-432c-a89f-966b45ad2329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:08.424525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec245c-277b-432c-a89f-966b45ad2329 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:08.424808 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec245c-277b-432c-a89f-966b45ad2329 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:08.425085 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec245c-277b-432c-a89f-966b45ad2329 None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:08.425763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ec245c-277b-432c-a89f-966b45ad2329 None admin] RBAC: Adding query filter params (name=service) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:08.426068 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ec245c-277b-432c-a89f-966b45ad2329 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:08.428146 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ec245c-277b-432c-a89f-966b45ad2329 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:08.444525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 299/581] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 12:31:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:08.444525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec245c-277b-432c-a89f-966b45ad2329 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:08.467884 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:08.516342 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e1300-ea84-495b-a09e-c251d8b4d035 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:08.555366 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cba8002-5404-469b-a53f-f20be437114f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:08.556010 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba8002-5404-469b-a53f-f20be437114f None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:08.556282 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba8002-5404-469b-a53f-f20be437114f None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:08.556557 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba8002-5404-469b-a53f-f20be437114f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:08.559420 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec245c-277b-432c-a89f-966b45ad2329 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:08.568850 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cba8002-5404-469b-a53f-f20be437114f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:08.570383 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cba8002-5404-469b-a53f-f20be437114f None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:08.578138 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:08.584946 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-8cba8002-5404-469b-a53f-f20be437114f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:08.584946 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:08.584946 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:08.584946 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:08.584946 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:08.584946 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:08.584946 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:08.584946 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:08.584946 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:08.584946 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:08.584946 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:08.584946 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:08.584946 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:08.584946 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:08.584946 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:08.584946 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:08.598808 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:08.598808 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:08.598808 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:08.598808 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:08.598808 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:08.598808 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:08.598808 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:08.598808 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:08.598808 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:08.598808 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:08.598808 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:08.598808 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:08.598808 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:08.598808 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:08.598808 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:08.598808 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:08.598808 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:08.600161 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:08.600161 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:08.600161 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:08.600161 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 283/582] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:08.600161 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cba8002-5404-469b-a53f-f20be437114f None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:08.601107 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a9742b-23e0-44f8-9deb-667de2623898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:08.601727 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a9742b-23e0-44f8-9deb-667de2623898 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:08.602007 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a9742b-23e0-44f8-9deb-667de2623898 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:08.602392 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a9742b-23e0-44f8-9deb-667de2623898 None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:08.603029 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a9742b-23e0-44f8-9deb-667de2623898 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:08.603358 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a9742b-23e0-44f8-9deb-667de2623898 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:08.604840 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a9742b-23e0-44f8-9deb-667de2623898 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:08.618245 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 300/583] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:08.619137 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a9742b-23e0-44f8-9deb-667de2623898 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:08.696337 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cba8002-5404-469b-a53f-f20be437114f None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:08.710406 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:08.721354 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a9742b-23e0-44f8-9deb-667de2623898 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:08.733370 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55da72d2-310e-460f-9a43-b4dda4b3da4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:08.734295 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da72d2-310e-460f-9a43-b4dda4b3da4f None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:08.734728 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da72d2-310e-460f-9a43-b4dda4b3da4f None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:08.735140 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da72d2-310e-460f-9a43-b4dda4b3da4f None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:08.735953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55da72d2-310e-460f-9a43-b4dda4b3da4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:08.736253 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55da72d2-310e-460f-9a43-b4dda4b3da4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:08.737603 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55da72d2-310e-460f-9a43-b4dda4b3da4f None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:08.750681 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 284/584] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:08.751536 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55da72d2-310e-460f-9a43-b4dda4b3da4f None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:08.759256 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c90fc-a3bf-473d-ac7b-d296faa98fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:08.760027 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c90fc-a3bf-473d-ac7b-d296faa98fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:08.760407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c90fc-a3bf-473d-ac7b-d296faa98fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:08.760778 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c90fc-a3bf-473d-ac7b-d296faa98fb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:08.773286 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376c90fc-a3bf-473d-ac7b-d296faa98fb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:08.774838 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376c90fc-a3bf-473d-ac7b-d296faa98fb0 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:08.791161 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-376c90fc-a3bf-473d-ac7b-d296faa98fb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:08.791161 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:08.791161 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:08.791161 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:08.791161 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:08.791161 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:08.791161 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:08.791161 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:08.791161 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:08.791161 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:08.791161 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:08.791161 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:08.791161 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:08.791161 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:08.791161 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:08.791161 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:08.792365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:08.792365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:08.792365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:08.792365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:08.792365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:08.792365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:08.792365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:08.792365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:08.792365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:08.792365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:08.792365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:08.792365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:08.792365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:08.792365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:08.792365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:08.792365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:08.792365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:08.793477 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:08.793477 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:08.793477 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:08.799233 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 301/585] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:08.799233 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c90fc-a3bf-473d-ac7b-d296faa98fb0 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:08.832692 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:08.862045 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55da72d2-310e-460f-9a43-b4dda4b3da4f None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:08.897326 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afcc026-f722-4e75-a99c-b3013be02ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:08.898306 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afcc026-f722-4e75-a99c-b3013be02ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:08.898585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afcc026-f722-4e75-a99c-b3013be02ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:08.898927 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afcc026-f722-4e75-a99c-b3013be02ff6 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:08.899605 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afcc026-f722-4e75-a99c-b3013be02ff6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:08.899903 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afcc026-f722-4e75-a99c-b3013be02ff6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:08.901263 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afcc026-f722-4e75-a99c-b3013be02ff6 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:08.909760 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c90fc-a3bf-473d-ac7b-d296faa98fb0 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:08.912935 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 285/586] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:08.913742 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afcc026-f722-4e75-a99c-b3013be02ff6 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:08.947222 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:08.954792 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e1bd51-4862-4ce7-a99f-34321e034920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:08.956069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1bd51-4862-4ce7-a99f-34321e034920 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:08.956519 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1bd51-4862-4ce7-a99f-34321e034920 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:08.964491 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1bd51-4862-4ce7-a99f-34321e034920 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:08.965358 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e1bd51-4862-4ce7-a99f-34321e034920 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:08.967788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e1bd51-4862-4ce7-a99f-34321e034920 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:08.981330 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-e3e1bd51-4862-4ce7-a99f-34321e034920 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:08.981330 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:08.981330 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:08.981330 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:08.981330 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:08.981330 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:08.981330 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:08.981330 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:08.981330 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:08.981330 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:08.981330 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:08.981330 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:08.981330 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 12:31:08.981330 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 12:31:08.981330 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 12:31:08.982192 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 12:31:08.982192 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:08.982192 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:08.982192 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:08.982192 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:08.982192 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:08.982192 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:08.982192 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:08.982192 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:08.982192 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:08.982192 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:08.982192 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:08.982192 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:08.982192 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:08.982192 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:08.982192 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:08.982192 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:08.983155 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 12:31:08.983155 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 12:31:08.983155 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 12:31:08.983155 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 12:31:08.983155 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 12:31:08.983155 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 12:31:08.983155 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:08.983155 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:08.984821 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 302/587] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:31:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:08.985759 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e1bd51-4862-4ce7-a99f-34321e034920 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:09.021580 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afcc026-f722-4e75-a99c-b3013be02ff6 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:09.056858 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967e6f95-82c1-40fb-9cbf-205c53208b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:09.057495 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967e6f95-82c1-40fb-9cbf-205c53208b21 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:09.057770 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967e6f95-82c1-40fb-9cbf-205c53208b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:09.058045 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967e6f95-82c1-40fb-9cbf-205c53208b21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:09.064793 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967e6f95-82c1-40fb-9cbf-205c53208b21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:09.066230 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967e6f95-82c1-40fb-9cbf-205c53208b21 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:09.078966 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-967e6f95-82c1-40fb-9cbf-205c53208b21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:09.078966 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:09.078966 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:09.078966 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:09.078966 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:09.078966 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:09.078966 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:09.078966 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:09.078966 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:09.078966 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:09.078966 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:09.078966 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:09.078966 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 12:31:09.078966 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 12:31:09.078966 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 12:31:09.080124 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 12:31:09.080124 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:09.080124 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:09.080124 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:09.080124 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:09.080124 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:09.080124 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:09.080124 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:09.080124 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:09.080124 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:09.080124 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:09.080124 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:09.080124 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:09.080124 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:09.080124 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:09.080124 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:09.080124 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:09.081016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 12:31:09.081016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 12:31:09.081016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 12:31:09.081016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 12:31:09.081016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 12:31:09.081016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 12:31:09.081016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:09.081016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:09.081016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 286/588] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:31:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:09.081016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967e6f95-82c1-40fb-9cbf-205c53208b21 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:09.088456 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:09.100687 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e1bd51-4862-4ce7-a99f-34321e034920 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:09.136086 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633575b0-89ed-4267-93b8-4fe981f898aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:09.136787 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633575b0-89ed-4267-93b8-4fe981f898aa None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:09.137158 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633575b0-89ed-4267-93b8-4fe981f898aa None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:09.137442 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633575b0-89ed-4267-93b8-4fe981f898aa None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:09.138096 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633575b0-89ed-4267-93b8-4fe981f898aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:09.138405 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633575b0-89ed-4267-93b8-4fe981f898aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:09.139788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633575b0-89ed-4267-93b8-4fe981f898aa None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:09.155364 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 303/589] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:09.155364 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633575b0-89ed-4267-93b8-4fe981f898aa None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:09.182134 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:09.190719 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967e6f95-82c1-40fb-9cbf-205c53208b21 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:09.227743 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9fdb53-8c2a-42f8-ac1c-1982d96a7030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:09.228494 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9fdb53-8c2a-42f8-ac1c-1982d96a7030 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:09.228844 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9fdb53-8c2a-42f8-ac1c-1982d96a7030 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:09.229182 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9fdb53-8c2a-42f8-ac1c-1982d96a7030 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:09.236670 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9fdb53-8c2a-42f8-ac1c-1982d96a7030 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:09.238413 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9fdb53-8c2a-42f8-ac1c-1982d96a7030 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:09.252045 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-1c9fdb53-8c2a-42f8-ac1c-1982d96a7030 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 03 12:31:09.252045 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:09.252045 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:09.252045 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:09.252045 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:09.252045 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:09.252045 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:09.252045 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:09.252045 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:09.252045 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:09.252045 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:09.252045 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:09.252045 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 12:31:09.252045 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 12:31:09.252045 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 12:31:09.252045 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 12:31:09.253030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:09.253030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:09.253030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 12:31:09.253030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:09.253030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 12:31:09.253030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:09.253030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:09.253030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:09.253030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:09.253030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:09.253030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:09.253030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:09.253030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:09.253030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:09.253030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:09.253030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:09.253030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:09.253030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:09.253984 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:09.253984 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:09.253984 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 12:31:09.253984 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 12:31:09.253984 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 12:31:09.253984 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 12:31:09.253984 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 12:31:09.253984 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 12:31:09.253984 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 03 12:31:09.253984 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:09.253984 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 287/590] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:31:09 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:09.254568 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9fdb53-8c2a-42f8-ac1c-1982d96a7030 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:09.271306 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633575b0-89ed-4267-93b8-4fe981f898aa None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:09.308542 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11512619-ac9e-4da7-9078-bda782cd87f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:09.309213 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11512619-ac9e-4da7-9078-bda782cd87f0 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:09.309404 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11512619-ac9e-4da7-9078-bda782cd87f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:09.309671 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11512619-ac9e-4da7-9078-bda782cd87f0 None admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:09.310350 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11512619-ac9e-4da7-9078-bda782cd87f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:09.310653 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11512619-ac9e-4da7-9078-bda782cd87f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:09.312081 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11512619-ac9e-4da7-9078-bda782cd87f0 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:09.324678 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 304/591] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 12:31:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:09.330480 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11512619-ac9e-4da7-9078-bda782cd87f0 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:09.342941 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:09.362172 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9fdb53-8c2a-42f8-ac1c-1982d96a7030 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:09.398722 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7260c067-b3ad-4d89-9c29-61b9e5614bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:09.399268 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260c067-b3ad-4d89-9c29-61b9e5614bbe None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:09.399578 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260c067-b3ad-4d89-9c29-61b9e5614bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:09.399842 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260c067-b3ad-4d89-9c29-61b9e5614bbe None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:09.400529 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7260c067-b3ad-4d89-9c29-61b9e5614bbe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:09.400832 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7260c067-b3ad-4d89-9c29-61b9e5614bbe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:09.402265 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7260c067-b3ad-4d89-9c29-61b9e5614bbe None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:09.417877 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 288/592] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 12:31:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:09.418700 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7260c067-b3ad-4d89-9c29-61b9e5614bbe None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:09.436350 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:09.449931 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11512619-ac9e-4da7-9078-bda782cd87f0 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:09.488146 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d9c51a-c466-45c8-9498-ae07f47540ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:09.488782 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9c51a-c466-45c8-9498-ae07f47540ae None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:09.489065 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9c51a-c466-45c8-9498-ae07f47540ae None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:09.489340 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9c51a-c466-45c8-9498-ae07f47540ae None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:09.496503 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d9c51a-c466-45c8-9498-ae07f47540ae None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:09.497923 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d9c51a-c466-45c8-9498-ae07f47540ae None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:09.509720 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-a1d9c51a-c466-45c8-9498-ae07f47540ae None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 03 12:31:09.509720 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:09.509720 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:09.509720 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:09.509720 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:09.509720 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:09.509720 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:09.509720 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:09.509720 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:09.509720 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:09.509720 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:09.509720 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:09.509720 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 12:31:09.509720 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 12:31:09.509720 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 12:31:09.509720 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 12:31:09.510845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:09.510845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:09.510845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 12:31:09.510845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:09.510845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 12:31:09.510845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:09.510845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:09.510845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:09.510845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:09.510845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:09.510845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:09.510845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:09.510845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:09.510845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:09.510845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:09.510845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:09.510845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:09.510845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:09.512003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:09.512003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:09.512003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 12:31:09.512003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 12:31:09.512003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 12:31:09.512003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 12:31:09.512003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 12:31:09.512003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 12:31:09.512003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 03 12:31:09.512003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:09.515488 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 305/593] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 12:31:09 2022] GET /identity/v3/users/neutron => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:09.516094 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d9c51a-c466-45c8-9498-ae07f47540ae None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:09.529078 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7260c067-b3ad-4d89-9c29-61b9e5614bbe None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:09.567119 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8772d154-69dc-43da-91c2-7ba5136ad44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:09.567798 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772d154-69dc-43da-91c2-7ba5136ad44a None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:09.568183 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772d154-69dc-43da-91c2-7ba5136ad44a None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:09.568559 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772d154-69dc-43da-91c2-7ba5136ad44a None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:09.569292 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8772d154-69dc-43da-91c2-7ba5136ad44a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:09.569592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8772d154-69dc-43da-91c2-7ba5136ad44a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:09.571067 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8772d154-69dc-43da-91c2-7ba5136ad44a None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:09.593759 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 289/594] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Oct 3 12:31:09 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:09.593759 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8772d154-69dc-43da-91c2-7ba5136ad44a None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:09.596196 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:09.632575 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d9c51a-c466-45c8-9498-ae07f47540ae None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:09.667920 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:09.670108 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf8139f-7d48-417a-a574-0a1e027243c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:09.670783 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf8139f-7d48-417a-a574-0a1e027243c0 None admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:09.671068 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf8139f-7d48-417a-a574-0a1e027243c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:09.671368 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf8139f-7d48-417a-a574-0a1e027243c0 None admin] PATH_INFO: `/v3/projects/4ea2fa8ad01240f7859182ebbe3051c7/users/524e9184eee1486cb071cad5c67050c9/roles/ff6b2d4413644db39a2bd8576afc6ccb` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:09.689089 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf8139f-7d48-417a-a574-0a1e027243c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=4ea2fa8ad01240f7859182ebbe3051c7, user_id=524e9184eee1486cb071cad5c67050c9, role_id=ff6b2d4413644db39a2bd8576afc6ccb)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:09.690631 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf8139f-7d48-417a-a574-0a1e027243c0 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:09.698549 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8772d154-69dc-43da-91c2-7ba5136ad44a None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:09.733027 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58407863-8cfd-4c22-ba9c-aa6ffcca2af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:09.733724 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58407863-8cfd-4c22-ba9c-aa6ffcca2af1 None admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:09.734005 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58407863-8cfd-4c22-ba9c-aa6ffcca2af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:09.734279 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58407863-8cfd-4c22-ba9c-aa6ffcca2af1 None admin] PATH_INFO: `/v3/projects/4ea2fa8ad01240f7859182ebbe3051c7/users/990f5837994f4a088c656c9b33e9a0dc/roles/ff6b2d4413644db39a2bd8576afc6ccb` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:09.748944 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58407863-8cfd-4c22-ba9c-aa6ffcca2af1 None admin] RBAC: Authorizing `identity:create_grant(project_id=4ea2fa8ad01240f7859182ebbe3051c7, user_id=990f5837994f4a088c656c9b33e9a0dc, role_id=ff6b2d4413644db39a2bd8576afc6ccb)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:09.750434 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58407863-8cfd-4c22-ba9c-aa6ffcca2af1 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:09.757525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 306/595] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 12:31:09 2022] PUT /identity/v3/projects/4ea2fa8ad01240f7859182ebbe3051c7/users/524e9184eee1486cb071cad5c67050c9/roles/ff6b2d4413644db39a2bd8576afc6ccb => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:31:09.758408 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf8139f-7d48-417a-a574-0a1e027243c0 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:09.839791 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 290/596] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 12:31:09 2022] PUT /identity/v3/projects/4ea2fa8ad01240f7859182ebbe3051c7/users/990f5837994f4a088c656c9b33e9a0dc/roles/ff6b2d4413644db39a2bd8576afc6ccb => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:31:09.841084 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58407863-8cfd-4c22-ba9c-aa6ffcca2af1 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:09.903915 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf8139f-7d48-417a-a574-0a1e027243c0 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:09.954382 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:09.976454 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b4cc3-a140-482b-9f10-714a432ef675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:09.977102 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b4cc3-a140-482b-9f10-714a432ef675 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:09.977490 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b4cc3-a140-482b-9f10-714a432ef675 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:09.977771 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b4cc3-a140-482b-9f10-714a432ef675 None admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:09.979366 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b4cc3-a140-482b-9f10-714a432ef675 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:09.979759 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b4cc3-a140-482b-9f10-714a432ef675 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:09.981332 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b4cc3-a140-482b-9f10-714a432ef675 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:10.009778 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 307/597] 10.222.0.33 () {56 vars in 1139 bytes} [Mon Oct 3 12:31:09 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:10.013586 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58407863-8cfd-4c22-ba9c-aa6ffcca2af1 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:10.015934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b4cc3-a140-482b-9f10-714a432ef675 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:10.053818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:10.075172 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c89d1b-7221-4559-b3c1-eb53008dff07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:10.076487 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c89d1b-7221-4559-b3c1-eb53008dff07 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:10.076937 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c89d1b-7221-4559-b3c1-eb53008dff07 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:10.077347 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c89d1b-7221-4559-b3c1-eb53008dff07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:10.097323 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c89d1b-7221-4559-b3c1-eb53008dff07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:10.098809 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c89d1b-7221-4559-b3c1-eb53008dff07 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:10.121192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-71c89d1b-7221-4559-b3c1-eb53008dff07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:10.121192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:10.121192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:10.121192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:10.121192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:10.121192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:10.121192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:10.121192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:10.121192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:10.121192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:10.121192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:10.121192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:10.121192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:10.121192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:10.121192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:10.121192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:10.122211 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:10.122211 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:10.122211 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:10.122211 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:10.122211 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:10.122211 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:10.122211 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:10.122211 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:10.122211 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:10.122211 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:10.122211 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:10.122211 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:10.122211 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:10.122211 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:10.122211 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:10.122211 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:10.122211 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:10.123649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:10.123649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:10.123649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:10.123649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 291/598] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:10.127767 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c89d1b-7221-4559-b3c1-eb53008dff07 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:10.171017 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b4cc3-a140-482b-9f10-714a432ef675 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:10.211038 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da836c75-1151-418d-ab33-233b09f58a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:10.211643 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da836c75-1151-418d-ab33-233b09f58a67 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:10.211929 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da836c75-1151-418d-ab33-233b09f58a67 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:10.212203 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da836c75-1151-418d-ab33-233b09f58a67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:10.248399 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da836c75-1151-418d-ab33-233b09f58a67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:10.249831 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da836c75-1151-418d-ab33-233b09f58a67 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:10.266976 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-da836c75-1151-418d-ab33-233b09f58a67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:10.266976 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:10.266976 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:10.266976 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:10.266976 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:10.266976 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:10.266976 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:10.266976 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:10.266976 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:10.266976 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:10.266976 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:10.266976 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:10.266976 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:10.266976 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:10.266976 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:10.266976 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:10.267990 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:10.267990 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:10.267990 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:10.267990 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:10.267990 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:10.267990 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:10.267990 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:10.267990 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:10.267990 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:10.267990 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:10.267990 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:10.267990 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:10.267990 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:10.267990 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:10.267990 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:10.267990 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:10.267990 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:10.269276 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:10.269276 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:10.269276 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:10.269276 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 308/599] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:31:10.274480 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da836c75-1151-418d-ab33-233b09f58a67 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:10.279688 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c89d1b-7221-4559-b3c1-eb53008dff07 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:10.318568 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d52d6-1c70-4fbd-92b6-28b91da29e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:10.319243 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d52d6-1c70-4fbd-92b6-28b91da29e34 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:10.319560 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d52d6-1c70-4fbd-92b6-28b91da29e34 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:10.319888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d52d6-1c70-4fbd-92b6-28b91da29e34 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:10.320538 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d52d6-1c70-4fbd-92b6-28b91da29e34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:10.320841 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d52d6-1c70-4fbd-92b6-28b91da29e34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:10.322280 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d52d6-1c70-4fbd-92b6-28b91da29e34 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:10.336304 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 292/600] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:31:10.341664 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d52d6-1c70-4fbd-92b6-28b91da29e34 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:10.348955 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:10.395642 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:10.484187 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da836c75-1151-418d-ab33-233b09f58a67 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:10.499452 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d52d6-1c70-4fbd-92b6-28b91da29e34 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:10.540780 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f82ca-08b0-4c0e-8efd-e8e13d0f70e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:10.541464 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f82ca-08b0-4c0e-8efd-e8e13d0f70e2 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:10.541747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f82ca-08b0-4c0e-8efd-e8e13d0f70e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:10.542023 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f82ca-08b0-4c0e-8efd-e8e13d0f70e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:10.550869 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f82ca-08b0-4c0e-8efd-e8e13d0f70e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:10.552842 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f82ca-08b0-4c0e-8efd-e8e13d0f70e2 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:10.566531 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-153f82ca-08b0-4c0e-8efd-e8e13d0f70e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:10.566531 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:10.566531 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:10.566531 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:10.566531 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:10.566531 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:10.566531 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:10.566531 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:10.566531 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:10.566531 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:10.566531 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:10.566531 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:10.566531 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 12:31:10.566531 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 12:31:10.566531 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 12:31:10.568791 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 12:31:10.568791 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:10.568791 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:10.568791 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:10.568791 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:10.568791 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:10.568791 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:10.568791 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:10.568791 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:10.568791 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:10.568791 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:10.568791 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:10.568791 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:10.568791 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:10.568791 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:10.568791 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:10.568791 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:10.569929 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 12:31:10.569929 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 12:31:10.569929 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 12:31:10.569929 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 12:31:10.569929 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 12:31:10.569929 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 12:31:10.569929 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:10.569929 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:10.569929 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 293/601] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:31:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:31:10.572079 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152d2d6-24a1-4af5-92d9-6603148d2423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:10.572834 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152d2d6-24a1-4af5-92d9-6603148d2423 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:10.573190 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152d2d6-24a1-4af5-92d9-6603148d2423 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:10.573538 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152d2d6-24a1-4af5-92d9-6603148d2423 None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:10.574142 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f82ca-08b0-4c0e-8efd-e8e13d0f70e2 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:10.574245 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5152d2d6-24a1-4af5-92d9-6603148d2423 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:10.574731 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5152d2d6-24a1-4af5-92d9-6603148d2423 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:10.576218 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5152d2d6-24a1-4af5-92d9-6603148d2423 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:10.617685 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 309/602] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:31:10.628365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152d2d6-24a1-4af5-92d9-6603148d2423 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:10.675582 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:10.685029 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f82ca-08b0-4c0e-8efd-e8e13d0f70e2 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:10.721860 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314541ab-ea6b-483a-b6db-9859b38087f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:10.722481 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314541ab-ea6b-483a-b6db-9859b38087f2 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:10.722761 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314541ab-ea6b-483a-b6db-9859b38087f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:10.723039 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314541ab-ea6b-483a-b6db-9859b38087f2 None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:10.723820 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314541ab-ea6b-483a-b6db-9859b38087f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:10.724784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314541ab-ea6b-483a-b6db-9859b38087f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:10.726223 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314541ab-ea6b-483a-b6db-9859b38087f2 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:10.738810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 294/603] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 12:31:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:31:10.744232 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d153b-3776-4167-a788-79b46f551f9b None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:10.744525 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d153b-3776-4167-a788-79b46f551f9b None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:10.744800 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d153b-3776-4167-a788-79b46f551f9b None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:10.746311 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 295/604] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:10.748076 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d153b-3776-4167-a788-79b46f551f9b None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:10.778372 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:10.791177 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152d2d6-24a1-4af5-92d9-6603148d2423 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:10.835505 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b90ed9-85cc-4c0a-ba8e-93e7e155dc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:10.836157 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b90ed9-85cc-4c0a-ba8e-93e7e155dc26 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:10.836439 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b90ed9-85cc-4c0a-ba8e-93e7e155dc26 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:10.836713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b90ed9-85cc-4c0a-ba8e-93e7e155dc26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:10.849863 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b90ed9-85cc-4c0a-ba8e-93e7e155dc26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:10.852465 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b90ed9-85cc-4c0a-ba8e-93e7e155dc26 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:10.871649 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-73b90ed9-85cc-4c0a-ba8e-93e7e155dc26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:10.871649 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:10.871649 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:10.871649 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:10.871649 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:10.871649 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:10.871649 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:10.871649 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:10.871649 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:10.871649 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:10.871649 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:10.871649 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:10.871649 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 12:31:10.871649 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 12:31:10.871649 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 12:31:10.872596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 12:31:10.872596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:10.872596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:10.872596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:10.872596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:10.872596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:10.872596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:10.872596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:10.872596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:10.872596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:10.872596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:10.872596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:10.872596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:10.872596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:10.872596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:10.872596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:10.872596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:10.874513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 12:31:10.874513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 12:31:10.874513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 12:31:10.874513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 12:31:10.874513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 12:31:10.874513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 12:31:10.874513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:10.874513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:10.874513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 310/605] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:31:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:10.874513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7bc5a-1a61-4f78-aa5e-d355eb429bf2 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:10.875744 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7bc5a-1a61-4f78-aa5e-d355eb429bf2 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:10.875744 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7bc5a-1a61-4f78-aa5e-d355eb429bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:10.875978 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d153b-3776-4167-a788-79b46f551f9b None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:10.898117 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-e4b7bc5a-1a61-4f78-aa5e-d355eb429bf2 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:10.899393 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-e4b7bc5a-1a61-4f78-aa5e-d355eb429bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:10.923221 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba6386a-1edd-4bf6-ac7c-be7caa1d7cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:10.924094 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6386a-1edd-4bf6-ac7c-be7caa1d7cf6 None admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:10.924500 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6386a-1edd-4bf6-ac7c-be7caa1d7cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:10.925257 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6386a-1edd-4bf6-ac7c-be7caa1d7cf6 None admin] PATH_INFO: `/v3/projects/4ea2fa8ad01240f7859182ebbe3051c7/users/b697e4847b13496d98a543675f5df1c4/roles/ff6b2d4413644db39a2bd8576afc6ccb` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:10.945511 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba6386a-1edd-4bf6-ac7c-be7caa1d7cf6 None admin] RBAC: Authorizing `identity:create_grant(project_id=4ea2fa8ad01240f7859182ebbe3051c7, user_id=b697e4847b13496d98a543675f5df1c4, role_id=ff6b2d4413644db39a2bd8576afc6ccb)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:10.946949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba6386a-1edd-4bf6-ac7c-be7caa1d7cf6 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:10.977860 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 311/606] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:31:10.978757 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7bc5a-1a61-4f78-aa5e-d355eb429bf2 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:11.079587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:11.099975 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7bc5a-1a61-4f78-aa5e-d355eb429bf2 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:11.134878 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e6fd6c-29ff-4c3c-b25d-d27bc259ff60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:11.135517 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6fd6c-29ff-4c3c-b25d-d27bc259ff60 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:11.135804 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6fd6c-29ff-4c3c-b25d-d27bc259ff60 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:11.136077 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6fd6c-29ff-4c3c-b25d-d27bc259ff60 None admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:11.136764 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e6fd6c-29ff-4c3c-b25d-d27bc259ff60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:11.137072 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e6fd6c-29ff-4c3c-b25d-d27bc259ff60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:11.138512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e6fd6c-29ff-4c3c-b25d-d27bc259ff60 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:11.151151 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 312/607] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 12:31:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:11.156471 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e6fd6c-29ff-4c3c-b25d-d27bc259ff60 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:11.261991 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e6fd6c-29ff-4c3c-b25d-d27bc259ff60 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:11.292136 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 296/608] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 12:31:10 2022] PUT /identity/v3/projects/4ea2fa8ad01240f7859182ebbe3051c7/users/b697e4847b13496d98a543675f5df1c4/roles/ff6b2d4413644db39a2bd8576afc6ccb => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:31:11.293591 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba6386a-1edd-4bf6-ac7c-be7caa1d7cf6 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:11.303107 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:11.306007 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4364938-46e1-4f31-ad4e-ac495c0eed45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:11.307285 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4364938-46e1-4f31-ad4e-ac495c0eed45 None admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:11.307820 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4364938-46e1-4f31-ad4e-ac495c0eed45 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:11.308344 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4364938-46e1-4f31-ad4e-ac495c0eed45 None admin] PATH_INFO: `/v3/projects/4ea2fa8ad01240f7859182ebbe3051c7/users/eb327a8a18ca414ab8b4111a4d13aad8/roles/ff6b2d4413644db39a2bd8576afc6ccb` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:11.325966 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4364938-46e1-4f31-ad4e-ac495c0eed45 None admin] RBAC: Authorizing `identity:create_grant(project_id=4ea2fa8ad01240f7859182ebbe3051c7, user_id=eb327a8a18ca414ab8b4111a4d13aad8, role_id=ff6b2d4413644db39a2bd8576afc6ccb)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:11.327957 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4364938-46e1-4f31-ad4e-ac495c0eed45 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:11.343818 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:11.409271 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 313/609] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 12:31:11 2022] PUT /identity/v3/projects/4ea2fa8ad01240f7859182ebbe3051c7/users/eb327a8a18ca414ab8b4111a4d13aad8/roles/ff6b2d4413644db39a2bd8576afc6ccb => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:31:11.450715 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba6386a-1edd-4bf6-ac7c-be7caa1d7cf6 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:11.503967 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b462150-9165-4110-84c8-577083a9ac96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:11.504789 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b462150-9165-4110-84c8-577083a9ac96 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:11.505159 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b462150-9165-4110-84c8-577083a9ac96 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:11.505632 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b462150-9165-4110-84c8-577083a9ac96 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:11.520810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b462150-9165-4110-84c8-577083a9ac96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:31:11.527145 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-5b462150-9165-4110-84c8-577083a9ac96 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:31:11.527145 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:11.527145 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:11.527145 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:11.527145 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:11.527145 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:11.527145 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:11.527145 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:11.527145 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:11.527145 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:11.527145 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:11.527145 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:11.527145 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:31:11.527145 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:31:11.527145 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:31:11.527145 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:31:11.528422 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:31:11.528422 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:31:11.528422 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:31:11.528422 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:31:11.528422 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:11.528422 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:11.528422 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:11.528422 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:11.528422 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:11.528422 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:11.528422 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:11.528422 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:11.528422 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:11.528422 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:11.528422 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:11.528422 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:11.529382 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:11.529382 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:11.529382 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:11.529382 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:11.529382 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:31:11.529382 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:31:11.529382 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:31:11.529382 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:31:11.529382 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:31:11.529382 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:31:11.529382 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:31:11.529382 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:11.530081 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 297/610] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 12:31:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:11.535794 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4364938-46e1-4f31-ad4e-ac495c0eed45 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:11.654082 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4364938-46e1-4f31-ad4e-ac495c0eed45 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:11.706806 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ff769-051e-444e-9ade-91e2a8cd940f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:11.707452 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ff769-051e-444e-9ade-91e2a8cd940f None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:11.707736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ff769-051e-444e-9ade-91e2a8cd940f None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:11.708009 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ff769-051e-444e-9ade-91e2a8cd940f None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:11.709295 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456ff769-051e-444e-9ade-91e2a8cd940f None admin] RBAC: Adding query filter params (name=service) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:11.709756 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456ff769-051e-444e-9ade-91e2a8cd940f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:11.711119 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456ff769-051e-444e-9ade-91e2a8cd940f None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:11.720234 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 314/611] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 12:31:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:11.727764 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b462150-9165-4110-84c8-577083a9ac96 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:11.816001 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:11.866124 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b462150-9165-4110-84c8-577083a9ac96 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:11.903258 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ddef4-bf37-4edd-9803-33fa7d2d34f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:11.903976 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ddef4-bf37-4edd-9803-33fa7d2d34f9 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:11.905663 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ddef4-bf37-4edd-9803-33fa7d2d34f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:11.905663 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ddef4-bf37-4edd-9803-33fa7d2d34f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:11.916463 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8ddef4-bf37-4edd-9803-33fa7d2d34f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:11.917987 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8ddef4-bf37-4edd-9803-33fa7d2d34f9 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:11.933706 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-aa8ddef4-bf37-4edd-9803-33fa7d2d34f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:11.933706 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:11.933706 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:11.933706 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:11.933706 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:11.933706 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:11.933706 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:11.933706 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:11.933706 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:11.933706 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:11.933706 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:11.933706 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:11.933706 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:11.933706 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:11.933706 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:11.933706 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:11.935515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:11.935515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:11.935515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:11.935515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:11.935515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:11.935515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:11.935515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:11.935515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:11.935515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:11.935515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:11.935515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:11.935515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:11.935515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:11.935515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:11.935515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:11.935515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:11.935515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:11.936632 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:11.936632 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:11.936632 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:11.936632 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 298/612] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:11.940750 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ff769-051e-444e-9ade-91e2a8cd940f None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:11.996524 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:12.044491 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ff769-051e-444e-9ade-91e2a8cd940f None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:12.091280 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62d9510-c4db-4203-8e18-0e81921a36e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:12.092176 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d9510-c4db-4203-8e18-0e81921a36e1 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:12.092463 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d9510-c4db-4203-8e18-0e81921a36e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:12.092734 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d9510-c4db-4203-8e18-0e81921a36e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:12.093379 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62d9510-c4db-4203-8e18-0e81921a36e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:12.093682 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62d9510-c4db-4203-8e18-0e81921a36e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:12.095044 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62d9510-c4db-4203-8e18-0e81921a36e1 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:12.110996 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 315/613] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:12.116252 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ddef4-bf37-4edd-9803-33fa7d2d34f9 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:12.282842 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ddef4-bf37-4edd-9803-33fa7d2d34f9 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:12.298115 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:12.322204 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd0d0a2-caaa-4f5d-928e-15e069700f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:12.322919 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd0d0a2-caaa-4f5d-928e-15e069700f6b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:12.323204 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd0d0a2-caaa-4f5d-928e-15e069700f6b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:12.323504 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd0d0a2-caaa-4f5d-928e-15e069700f6b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:12.331605 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd0d0a2-caaa-4f5d-928e-15e069700f6b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:12.333134 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd0d0a2-caaa-4f5d-928e-15e069700f6b None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:12.345904 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-7bd0d0a2-caaa-4f5d-928e-15e069700f6b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 03 12:31:12.345904 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:12.345904 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:12.345904 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:12.345904 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:12.345904 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:12.345904 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:12.345904 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:12.345904 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:12.345904 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:12.345904 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:12.345904 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:12.345904 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 12:31:12.345904 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 12:31:12.345904 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 12:31:12.345904 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 12:31:12.346878 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:12.346878 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:12.346878 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 12:31:12.346878 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:12.346878 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 12:31:12.346878 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:12.346878 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:12.346878 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:12.346878 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:12.346878 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:12.346878 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:12.346878 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:12.346878 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:12.346878 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:12.346878 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:12.346878 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:12.346878 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:12.346878 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:12.347892 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:12.347892 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:12.347892 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 12:31:12.347892 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 12:31:12.347892 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 12:31:12.347892 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 12:31:12.347892 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 12:31:12.347892 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 12:31:12.347892 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 03 12:31:12.347892 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:12.347892 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 299/614] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 12:31:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:12.353266 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62d9510-c4db-4203-8e18-0e81921a36e1 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:12.420241 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f16c36-b7b1-4ec2-8e7b-1a48be2b54c0 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:12.420457 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f16c36-b7b1-4ec2-8e7b-1a48be2b54c0 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:12.420716 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f16c36-b7b1-4ec2-8e7b-1a48be2b54c0 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:12.422698 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 300/615] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:12.430653 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ffec31-7be7-4a99-9e13-127565552499 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:12.430937 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ffec31-7be7-4a99-9e13-127565552499 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:12.431218 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ffec31-7be7-4a99-9e13-127565552499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:12.460719 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-12ffec31-7be7-4a99-9e13-127565552499 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:12.461966 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-12ffec31-7be7-4a99-9e13-127565552499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:12.552970 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62d9510-c4db-4203-8e18-0e81921a36e1 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:12.565119 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:12.592467 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b238c0-31c5-44c7-8efb-da8912a006db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:12.593082 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b238c0-31c5-44c7-8efb-da8912a006db None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:12.593364 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b238c0-31c5-44c7-8efb-da8912a006db None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:12.593635 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b238c0-31c5-44c7-8efb-da8912a006db None admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:12.594373 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b238c0-31c5-44c7-8efb-da8912a006db None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:12.594678 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b238c0-31c5-44c7-8efb-da8912a006db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:12.597404 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b238c0-31c5-44c7-8efb-da8912a006db None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:12.607262 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 301/616] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 12:31:12.612512 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa15023-5f9a-4e4c-a8b8-6aa1217d4a17 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:12.612795 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa15023-5f9a-4e4c-a8b8-6aa1217d4a17 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:12.613064 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa15023-5f9a-4e4c-a8b8-6aa1217d4a17 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:12.614330 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 302/617] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 12:31:12.626008 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f610e8-9038-49f7-9649-ca3fccba60ae None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:12.626286 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f610e8-9038-49f7-9649-ca3fccba60ae None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:12.626555 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f610e8-9038-49f7-9649-ca3fccba60ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:12.633812 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 316/618] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Oct 3 12:31:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:12.639441 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b238c0-31c5-44c7-8efb-da8912a006db None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:12.658026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-94f610e8-9038-49f7-9649-ca3fccba60ae None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:12.659896 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-94f610e8-9038-49f7-9649-ca3fccba60ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:12.805507 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 303/619] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 12:31:12.808363 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b238c0-31c5-44c7-8efb-da8912a006db None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:12.844851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb1ed10-e2a1-4655-8c0d-fc54a7d71a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:12.845486 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb1ed10-e2a1-4655-8c0d-fc54a7d71a1d None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:12.845769 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb1ed10-e2a1-4655-8c0d-fc54a7d71a1d None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:12.846043 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb1ed10-e2a1-4655-8c0d-fc54a7d71a1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:12.859756 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb1ed10-e2a1-4655-8c0d-fc54a7d71a1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:12.861193 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb1ed10-e2a1-4655-8c0d-fc54a7d71a1d None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:12.877176 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-7cb1ed10-e2a1-4655-8c0d-fc54a7d71a1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:12.877176 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:12.877176 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:12.877176 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:12.877176 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:12.877176 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:12.877176 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:12.877176 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:12.877176 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:12.877176 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:12.877176 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:12.877176 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:12.877176 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:12.877176 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:12.877176 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:12.877176 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:12.879055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:12.879055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:12.879055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:12.879055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:12.879055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:12.879055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:12.879055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:12.879055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:12.879055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:12.879055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:12.879055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:12.879055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:12.879055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:12.879055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:12.879055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:12.879055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:12.879055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:12.881564 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:12.881564 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:12.881564 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:12.881564 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 317/620] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:31:12.884790 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f610e8-9038-49f7-9649-ca3fccba60ae None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:12.928338 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb1ed10-e2a1-4655-8c0d-fc54a7d71a1d None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:12.946384 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:12.968333 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:13.064576 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f610e8-9038-49f7-9649-ca3fccba60ae None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:13.109873 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0a661-1f80-4504-8cc1-12db66b90656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:13.111902 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0a661-1f80-4504-8cc1-12db66b90656 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:13.112184 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0a661-1f80-4504-8cc1-12db66b90656 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:13.112459 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0a661-1f80-4504-8cc1-12db66b90656 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:13.113105 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace0a661-1f80-4504-8cc1-12db66b90656 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:13.113406 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace0a661-1f80-4504-8cc1-12db66b90656 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:13.114779 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace0a661-1f80-4504-8cc1-12db66b90656 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:13.122936 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb1ed10-e2a1-4655-8c0d-fc54a7d71a1d None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:13.137203 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 304/621] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:13.138127 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0a661-1f80-4504-8cc1-12db66b90656 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:13.168778 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77593f-d4fb-470d-aa0f-3eaca36cba58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:13.169807 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77593f-d4fb-470d-aa0f-3eaca36cba58 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:13.170193 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77593f-d4fb-470d-aa0f-3eaca36cba58 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:13.170588 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77593f-d4fb-470d-aa0f-3eaca36cba58 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:13.182346 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77593f-d4fb-470d-aa0f-3eaca36cba58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:31:13.187722 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-0d77593f-d4fb-470d-aa0f-3eaca36cba58 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:31:13.187722 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:13.187722 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:13.187722 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:13.187722 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:13.187722 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:13.187722 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:13.187722 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:13.187722 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:13.187722 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:13.187722 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:13.187722 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:13.187722 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:31:13.187722 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:31:13.187722 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:31:13.187722 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:31:13.188878 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:31:13.188878 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:31:13.188878 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:31:13.188878 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:31:13.188878 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:13.188878 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:13.188878 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:13.188878 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:13.188878 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:13.188878 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:13.188878 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:13.188878 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:13.188878 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:13.188878 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:13.188878 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:13.188878 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:13.189921 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:13.189921 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:13.189921 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:13.189921 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:13.189921 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:31:13.189921 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:31:13.189921 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:31:13.189921 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:31:13.189921 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:31:13.189921 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:31:13.189921 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:31:13.189921 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:13.191731 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 318/622] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 12:31:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:31:13.192449 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77593f-d4fb-470d-aa0f-3eaca36cba58 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:13.273530 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:13.280935 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0a661-1f80-4504-8cc1-12db66b90656 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:13.303831 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:13.324594 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77593f-d4fb-470d-aa0f-3eaca36cba58 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:13.327927 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d3c37b-51fa-43c1-912c-1b570195b694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:13.328571 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3c37b-51fa-43c1-912c-1b570195b694 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:13.328847 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3c37b-51fa-43c1-912c-1b570195b694 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:13.329115 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3c37b-51fa-43c1-912c-1b570195b694 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:13.344845 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d3c37b-51fa-43c1-912c-1b570195b694 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:31:13.353192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-a0d3c37b-51fa-43c1-912c-1b570195b694 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:31:13.353192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:13.353192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:13.353192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:13.353192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:13.353192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:13.353192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:13.353192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:13.353192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:13.353192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:13.353192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:13.353192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:13.353192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:31:13.353192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:31:13.353192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:31:13.353192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:31:13.354118 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:31:13.354118 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:31:13.354118 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:31:13.354118 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:31:13.354118 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:13.354118 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:13.354118 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:13.354118 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:13.354118 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:13.354118 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:13.354118 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:13.354118 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:13.354118 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:13.354118 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:13.354118 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:13.354118 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:13.354936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:13.354936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:13.354936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:13.354936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:13.354936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:31:13.354936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:31:13.354936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:31:13.354936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:31:13.354936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:31:13.354936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:31:13.354936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:31:13.354936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:13.361812 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 305/623] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 12:31:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:13.361812 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d3c37b-51fa-43c1-912c-1b570195b694 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:13.375606 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330bafa9-9b9a-4195-a272-234a0ca116c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:13.376724 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330bafa9-9b9a-4195-a272-234a0ca116c5 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:13.377592 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330bafa9-9b9a-4195-a272-234a0ca116c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:13.378066 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330bafa9-9b9a-4195-a272-234a0ca116c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:13.386975 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330bafa9-9b9a-4195-a272-234a0ca116c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:13.389255 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330bafa9-9b9a-4195-a272-234a0ca116c5 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:13.404398 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-330bafa9-9b9a-4195-a272-234a0ca116c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:13.404398 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:13.404398 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:13.404398 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:13.404398 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:13.404398 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:13.404398 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:13.404398 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:13.404398 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:13.404398 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:13.404398 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:13.404398 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:13.404398 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 12:31:13.404398 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 12:31:13.404398 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 12:31:13.405304 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 12:31:13.405304 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:13.405304 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:13.405304 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:13.405304 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:13.405304 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:13.405304 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:13.405304 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:13.405304 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:13.405304 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:13.405304 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:13.405304 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:13.405304 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:13.405304 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:13.405304 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:13.405304 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:13.405304 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:13.406168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 12:31:13.406168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 12:31:13.406168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 12:31:13.406168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 12:31:13.406168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 12:31:13.406168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 12:31:13.406168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:13.406168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:13.411623 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 319/624] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:31:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:13.413377 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330bafa9-9b9a-4195-a272-234a0ca116c5 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:13.485548 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d3c37b-51fa-43c1-912c-1b570195b694 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:13.523178 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330bafa9-9b9a-4195-a272-234a0ca116c5 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:13.529159 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8971c5fb-8d8e-42bb-9e45-8ad338ea9fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:13.530067 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8971c5fb-8d8e-42bb-9e45-8ad338ea9fab None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:13.530557 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8971c5fb-8d8e-42bb-9e45-8ad338ea9fab None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:13.531059 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8971c5fb-8d8e-42bb-9e45-8ad338ea9fab None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:13.533715 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8971c5fb-8d8e-42bb-9e45-8ad338ea9fab None admin] RBAC: Adding query filter params (name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:13.534021 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8971c5fb-8d8e-42bb-9e45-8ad338ea9fab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:13.535414 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8971c5fb-8d8e-42bb-9e45-8ad338ea9fab None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:13.544546 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 306/625] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 12:31:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:13.545462 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8971c5fb-8d8e-42bb-9e45-8ad338ea9fab None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:13.566486 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:13.567937 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981976b9-2a40-40af-af9a-361915a1f146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:13.570326 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981976b9-2a40-40af-af9a-361915a1f146 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:13.570821 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981976b9-2a40-40af-af9a-361915a1f146 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:13.571293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981976b9-2a40-40af-af9a-361915a1f146 None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:13.572182 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981976b9-2a40-40af-af9a-361915a1f146 None admin] RBAC: Adding query filter params (name=service) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:13.573197 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981976b9-2a40-40af-af9a-361915a1f146 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:13.575330 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981976b9-2a40-40af-af9a-361915a1f146 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:13.593591 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 320/626] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 12:31:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:13.594839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981976b9-2a40-40af-af9a-361915a1f146 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:13.600139 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:13.663544 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8971c5fb-8d8e-42bb-9e45-8ad338ea9fab None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:13.698703 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16b260-55d3-4e49-af25-293bebb766d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:13.699674 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16b260-55d3-4e49-af25-293bebb766d3 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:13.699808 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16b260-55d3-4e49-af25-293bebb766d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:13.699993 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16b260-55d3-4e49-af25-293bebb766d3 None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:13.700679 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f16b260-55d3-4e49-af25-293bebb766d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:13.700978 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f16b260-55d3-4e49-af25-293bebb766d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:13.702461 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f16b260-55d3-4e49-af25-293bebb766d3 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:13.720208 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 307/627] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 12:31:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:13.721090 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16b260-55d3-4e49-af25-293bebb766d3 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:13.731809 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981976b9-2a40-40af-af9a-361915a1f146 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:13.779839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d94e1-fea7-4bc7-be03-f6fbbcbaf461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:13.781071 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d94e1-fea7-4bc7-be03-f6fbbcbaf461 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:13.781806 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d94e1-fea7-4bc7-be03-f6fbbcbaf461 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:13.782671 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d94e1-fea7-4bc7-be03-f6fbbcbaf461 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:13.798237 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809d94e1-fea7-4bc7-be03-f6fbbcbaf461 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:13.800081 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809d94e1-fea7-4bc7-be03-f6fbbcbaf461 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:13.805853 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:13.817989 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-809d94e1-fea7-4bc7-be03-f6fbbcbaf461 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:13.817989 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:13.817989 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:13.817989 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:13.817989 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:13.817989 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:13.817989 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:13.817989 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:13.817989 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:13.817989 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:13.817989 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:13.817989 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:13.817989 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:13.817989 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:13.817989 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:13.817989 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:13.825390 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:13.825390 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:13.825390 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:13.825390 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:13.825390 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:13.825390 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:13.825390 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:13.825390 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:13.825390 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:13.825390 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:13.825390 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:13.825390 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:13.825390 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:13.825390 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:13.825390 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:13.825390 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:13.825390 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:13.826341 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:13.826341 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:13.826341 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:13.826341 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 321/628] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:13.826341 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d94e1-fea7-4bc7-be03-f6fbbcbaf461 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:13.836078 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16b260-55d3-4e49-af25-293bebb766d3 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:13.874514 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b4eb3-e45b-4de9-87c7-27b7482fbbab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:13.875349 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b4eb3-e45b-4de9-87c7-27b7482fbbab None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:13.875669 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b4eb3-e45b-4de9-87c7-27b7482fbbab None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:13.876012 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b4eb3-e45b-4de9-87c7-27b7482fbbab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:13.888493 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:13.890058 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b4eb3-e45b-4de9-87c7-27b7482fbbab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:13.891744 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b4eb3-e45b-4de9-87c7-27b7482fbbab None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:13.908774 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-4e3b4eb3-e45b-4de9-87c7-27b7482fbbab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:13.908774 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:13.908774 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:13.908774 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:13.908774 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:13.908774 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:13.908774 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:13.908774 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:13.908774 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:13.908774 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:13.908774 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:13.908774 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:13.908774 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:13.908774 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:13.908774 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:13.908774 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:13.909767 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:13.909767 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:13.909767 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:13.909767 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:13.909767 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:13.909767 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:13.909767 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:13.909767 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:13.909767 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:13.909767 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:13.909767 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:13.909767 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:13.909767 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:13.909767 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:13.909767 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:13.909767 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:13.909767 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:13.910640 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:13.910640 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:13.910640 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:13.914930 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 308/629] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:13.915836 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b4eb3-e45b-4de9-87c7-27b7482fbbab None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:13.942266 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d94e1-fea7-4bc7-be03-f6fbbcbaf461 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:13.979506 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b05074-e038-43d8-912c-bfd0298c8b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:13.980558 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b05074-e038-43d8-912c-bfd0298c8b5e None admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:13.981054 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b05074-e038-43d8-912c-bfd0298c8b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:13.981531 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b05074-e038-43d8-912c-bfd0298c8b5e None admin] PATH_INFO: `/v3/projects/4ea2fa8ad01240f7859182ebbe3051c7/users/b111d72a34144073917de4f566a58233/roles/ff6b2d4413644db39a2bd8576afc6ccb` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:13.998823 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b05074-e038-43d8-912c-bfd0298c8b5e None admin] RBAC: Authorizing `identity:create_grant(project_id=4ea2fa8ad01240f7859182ebbe3051c7, user_id=b111d72a34144073917de4f566a58233, role_id=ff6b2d4413644db39a2bd8576afc6ccb)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:14.000310 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b05074-e038-43d8-912c-bfd0298c8b5e None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:14.023447 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b4eb3-e45b-4de9-87c7-27b7482fbbab None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:14.049314 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:14.056893 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec03082-1c0b-4c30-9aca-9b08be72e00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:14.057510 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec03082-1c0b-4c30-9aca-9b08be72e00b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:14.057788 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec03082-1c0b-4c30-9aca-9b08be72e00b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:14.058059 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec03082-1c0b-4c30-9aca-9b08be72e00b None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:14.058696 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec03082-1c0b-4c30-9aca-9b08be72e00b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:14.058997 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec03082-1c0b-4c30-9aca-9b08be72e00b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:14.060436 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec03082-1c0b-4c30-9aca-9b08be72e00b None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:14.079662 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 322/630] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 12:31:13 2022] PUT /identity/v3/projects/4ea2fa8ad01240f7859182ebbe3051c7/users/b111d72a34144073917de4f566a58233/roles/ff6b2d4413644db39a2bd8576afc6ccb => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:31:14.079662 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b05074-e038-43d8-912c-bfd0298c8b5e None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:14.085633 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 309/631] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:14.087386 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61b35e-8ba6-4d2c-ae37-4d57a66cefb3 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:14.087691 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61b35e-8ba6-4d2c-ae37-4d57a66cefb3 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:14.087963 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61b35e-8ba6-4d2c-ae37-4d57a66cefb3 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:14.089837 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 310/632] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:14.097894 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe61b35e-8ba6-4d2c-ae37-4d57a66cefb3 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:14.217344 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b05074-e038-43d8-912c-bfd0298c8b5e None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:14.217836 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe61b35e-8ba6-4d2c-ae37-4d57a66cefb3 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:14.228098 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:14.257720 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70dca1e-c5c8-43ba-949c-207c43eb56c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:14.258400 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70dca1e-c5c8-43ba-949c-207c43eb56c2 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:14.258676 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70dca1e-c5c8-43ba-949c-207c43eb56c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:14.258946 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70dca1e-c5c8-43ba-949c-207c43eb56c2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:14.264577 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7d92b-3821-4354-a26e-75f584c40e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:14.265208 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7d92b-3821-4354-a26e-75f584c40e1a None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:14.265486 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7d92b-3821-4354-a26e-75f584c40e1a None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:14.265760 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7d92b-3821-4354-a26e-75f584c40e1a None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:14.269607 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7d92b-3821-4354-a26e-75f584c40e1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:14.269607 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7d92b-3821-4354-a26e-75f584c40e1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:14.269793 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70dca1e-c5c8-43ba-949c-207c43eb56c2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:14.269881 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7d92b-3821-4354-a26e-75f584c40e1a None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:14.269951 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70dca1e-c5c8-43ba-949c-207c43eb56c2 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:14.284495 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 323/633] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:14.285328 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-d70dca1e-c5c8-43ba-949c-207c43eb56c2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 03 12:31:14.285328 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:14.285328 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:14.285328 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:14.285328 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:14.285328 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:14.285328 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:14.285328 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:14.285328 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:14.285328 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:14.285328 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:14.285328 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:14.285328 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 12:31:14.285328 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 12:31:14.285328 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 12:31:14.285328 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 12:31:14.286221 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:14.286221 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:14.286221 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 12:31:14.286221 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:14.286221 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 12:31:14.286221 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:14.286221 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:14.286221 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:14.286221 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:14.286221 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:14.286221 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:14.286221 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:14.286221 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:14.286221 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:14.286221 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:14.286221 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:14.286221 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:14.286221 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:14.287864 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:14.287864 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:14.287864 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 12:31:14.287864 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 12:31:14.287864 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 12:31:14.287864 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 12:31:14.287864 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 12:31:14.287864 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 12:31:14.287864 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 03 12:31:14.287864 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:14.288407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347c460-22b0-4ab6-8471-0674566cec92 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:14.288407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347c460-22b0-4ab6-8471-0674566cec92 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:14.288407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347c460-22b0-4ab6-8471-0674566cec92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:14.288598 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 311/634] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:31:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:14.288598 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa75592-8274-4dac-88ab-a66af6f0ee0f None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:14.289012 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa75592-8274-4dac-88ab-a66af6f0ee0f None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:14.289106 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa75592-8274-4dac-88ab-a66af6f0ee0f None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:14.290656 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 312/635] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:14.292061 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa75592-8274-4dac-88ab-a66af6f0ee0f None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:14.309988 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-f347c460-22b0-4ab6-8471-0674566cec92 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:14.311170 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-f347c460-22b0-4ab6-8471-0674566cec92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:14.346395 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:14.382573 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 324/636] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:31:14.383406 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347c460-22b0-4ab6-8471-0674566cec92 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:14.408721 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa75592-8274-4dac-88ab-a66af6f0ee0f None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:14.448889 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4915fdd-2c2f-4584-93ac-40878831c85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:14.449522 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4915fdd-2c2f-4584-93ac-40878831c85b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:14.449791 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4915fdd-2c2f-4584-93ac-40878831c85b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:14.450063 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4915fdd-2c2f-4584-93ac-40878831c85b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:14.457368 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4915fdd-2c2f-4584-93ac-40878831c85b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:14.459583 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4915fdd-2c2f-4584-93ac-40878831c85b None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:14.471512 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-c4915fdd-2c2f-4584-93ac-40878831c85b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 03 12:31:14.471512 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:14.471512 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:14.471512 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:14.471512 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:14.471512 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:14.471512 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:14.471512 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:14.471512 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:14.471512 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:14.471512 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:14.471512 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:14.471512 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 12:31:14.471512 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 12:31:14.471512 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 12:31:14.471512 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 12:31:14.472476 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:14.472476 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:14.472476 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 12:31:14.472476 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:14.472476 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 12:31:14.472476 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:14.472476 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:14.472476 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:14.472476 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:14.472476 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:14.472476 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:14.472476 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:14.472476 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:14.472476 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:14.472476 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:14.472476 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:14.472476 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:14.472476 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:14.473464 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:14.473464 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:14.473464 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 12:31:14.473464 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 12:31:14.473464 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 12:31:14.473464 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 12:31:14.473464 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 12:31:14.473464 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 12:31:14.473464 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 03 12:31:14.473464 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:14.473464 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 313/637] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Oct 3 12:31:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:31:14.474064 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea21bd-b767-467b-b73a-07b237495dcd None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:14.474688 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea21bd-b767-467b-b73a-07b237495dcd None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:14.474688 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea21bd-b767-467b-b73a-07b237495dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:14.494711 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-ebea21bd-b767-467b-b73a-07b237495dcd None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:14.496037 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ebea21bd-b767-467b-b73a-07b237495dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:14.502534 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:14.509875 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347c460-22b0-4ab6-8471-0674566cec92 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:14.561114 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b3ab5e-96c5-4788-8285-1a9e4f397ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:14.561733 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3ab5e-96c5-4788-8285-1a9e4f397ed2 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:14.562011 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3ab5e-96c5-4788-8285-1a9e4f397ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:14.562277 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3ab5e-96c5-4788-8285-1a9e4f397ed2 None admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:14.563007 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b3ab5e-96c5-4788-8285-1a9e4f397ed2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:14.563310 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b3ab5e-96c5-4788-8285-1a9e4f397ed2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:14.564747 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b3ab5e-96c5-4788-8285-1a9e4f397ed2 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:14.577525 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 314/638] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:31:14.578285 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea21bd-b767-467b-b73a-07b237495dcd None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:14.588982 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 325/639] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Oct 3 12:31:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:14.598136 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b3ab5e-96c5-4788-8285-1a9e4f397ed2 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:14.647607 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:14.738327 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea21bd-b767-467b-b73a-07b237495dcd None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:14.742628 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b3ab5e-96c5-4788-8285-1a9e4f397ed2 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:14.779264 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8e5cf4-496f-43b8-a25d-18c5a2b8908d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:14.779904 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8e5cf4-496f-43b8-a25d-18c5a2b8908d None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:14.780184 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8e5cf4-496f-43b8-a25d-18c5a2b8908d None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:14.780455 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8e5cf4-496f-43b8-a25d-18c5a2b8908d None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:14.781178 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8e5cf4-496f-43b8-a25d-18c5a2b8908d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:14.781478 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8e5cf4-496f-43b8-a25d-18c5a2b8908d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:14.782914 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8e5cf4-496f-43b8-a25d-18c5a2b8908d None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:14.783042 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d8a65-0ecd-451c-a0f7-5980c8ea5b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:14.783682 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d8a65-0ecd-451c-a0f7-5980c8ea5b60 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:14.784028 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d8a65-0ecd-451c-a0f7-5980c8ea5b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:14.784363 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d8a65-0ecd-451c-a0f7-5980c8ea5b60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:14.806453 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 315/640] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Oct 3 12:31:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:14.807390 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8e5cf4-496f-43b8-a25d-18c5a2b8908d None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:14.811855 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:14.864387 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05d8a65-0ecd-451c-a0f7-5980c8ea5b60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:14.866649 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05d8a65-0ecd-451c-a0f7-5980c8ea5b60 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:14.909551 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:14.916059 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8e5cf4-496f-43b8-a25d-18c5a2b8908d None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:14.960351 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e17032-d648-43df-9da0-6b93e6847da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:14.961293 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e17032-d648-43df-9da0-6b93e6847da9 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:14.961697 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e17032-d648-43df-9da0-6b93e6847da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:14.961979 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e17032-d648-43df-9da0-6b93e6847da9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:14.973924 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e17032-d648-43df-9da0-6b93e6847da9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:31:14.979882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-21e17032-d648-43df-9da0-6b93e6847da9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:31:14.979882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:14.979882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:14.979882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:14.979882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:14.979882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:14.979882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:14.979882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:14.979882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:14.979882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:14.979882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:14.979882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:14.979882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:31:14.979882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:31:14.979882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:31:14.979882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:31:14.980799 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:31:14.980799 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:31:14.980799 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:31:14.980799 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:31:14.980799 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:14.980799 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:14.980799 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:14.980799 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:14.980799 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:14.980799 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:14.980799 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:14.980799 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:14.980799 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:14.980799 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:14.980799 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:14.980799 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:14.981629 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:14.981629 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:14.981629 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:14.981629 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:14.981629 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:31:14.981629 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:31:14.981629 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:31:14.981629 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:31:14.981629 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:31:14.981629 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:31:14.981629 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:31:14.981629 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:14.981629 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 316/641] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 12:31:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:14.982409 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 326/642] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:31:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 12:31:14.982484 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e17032-d648-43df-9da0-6b93e6847da9 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:14.982560 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d8a65-0ecd-451c-a0f7-5980c8ea5b60 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:15.044986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:15.090121 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e17032-d648-43df-9da0-6b93e6847da9 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:15.098881 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d8a65-0ecd-451c-a0f7-5980c8ea5b60 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:15.124635 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21446d42-e6de-4777-92c8-e9f288f07562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:15.125194 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21446d42-e6de-4777-92c8-e9f288f07562 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:15.125444 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21446d42-e6de-4777-92c8-e9f288f07562 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:15.125719 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21446d42-e6de-4777-92c8-e9f288f07562 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:15.138717 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd14160-c6fb-4515-ae48-045b437b1b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:15.139868 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd14160-c6fb-4515-ae48-045b437b1b69 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:15.140381 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd14160-c6fb-4515-ae48-045b437b1b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:15.140842 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd14160-c6fb-4515-ae48-045b437b1b69 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:15.151087 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:15.153122 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd14160-c6fb-4515-ae48-045b437b1b69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:31:15.159239 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-afd14160-c6fb-4515-ae48-045b437b1b69 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:31:15.159239 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:15.159239 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:15.159239 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:15.159239 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:15.159239 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:15.159239 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:15.159239 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:15.159239 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:15.159239 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:15.159239 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:15.159239 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:15.159239 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:31:15.159239 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:31:15.159239 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:31:15.159239 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:31:15.160259 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:31:15.160259 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:31:15.160259 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:31:15.160259 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:31:15.160259 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:15.160259 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:15.160259 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:15.160259 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:15.160259 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:15.160259 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:15.160259 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:15.160259 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:15.160259 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:15.160259 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:15.160259 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:15.160259 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:15.167098 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:15.167098 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:15.167098 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:15.167098 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:15.167098 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:31:15.167098 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:31:15.167098 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:31:15.167098 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:31:15.167098 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:31:15.167098 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:31:15.167098 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:31:15.167098 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:15.167098 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 327/643] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 12:31:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:15.167098 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd14160-c6fb-4515-ae48-045b437b1b69 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:15.195041 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21446d42-e6de-4777-92c8-e9f288f07562 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:15.196702 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21446d42-e6de-4777-92c8-e9f288f07562 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:15.280571 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd14160-c6fb-4515-ae48-045b437b1b69 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:15.294378 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:15.315535 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec67b3-4784-4a2e-86e3-b783ed7e413f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:15.316142 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec67b3-4784-4a2e-86e3-b783ed7e413f None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:15.316487 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec67b3-4784-4a2e-86e3-b783ed7e413f None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:15.316763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec67b3-4784-4a2e-86e3-b783ed7e413f None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:15.317430 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec67b3-4784-4a2e-86e3-b783ed7e413f None admin] RBAC: Adding query filter params (name=service) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:15.317738 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec67b3-4784-4a2e-86e3-b783ed7e413f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:15.319729 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec67b3-4784-4a2e-86e3-b783ed7e413f None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:15.321494 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 317/644] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:31:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 12:31:15.326689 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21446d42-e6de-4777-92c8-e9f288f07562 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:15.333592 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 328/645] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 12:31:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:15.333592 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec67b3-4784-4a2e-86e3-b783ed7e413f None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:15.389083 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:15.434606 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec67b3-4784-4a2e-86e3-b783ed7e413f None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:15.436386 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21446d42-e6de-4777-92c8-e9f288f07562 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:15.470157 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109010b8-714f-472e-bd2a-1e10406cc354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:15.470616 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c24dfa8-91b4-4422-864d-20f2492af276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:15.470805 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109010b8-714f-472e-bd2a-1e10406cc354 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:15.471044 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109010b8-714f-472e-bd2a-1e10406cc354 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:15.471245 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24dfa8-91b4-4422-864d-20f2492af276 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:15.471437 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109010b8-714f-472e-bd2a-1e10406cc354 None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:15.471613 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24dfa8-91b4-4422-864d-20f2492af276 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:15.471817 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24dfa8-91b4-4422-864d-20f2492af276 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:15.471995 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109010b8-714f-472e-bd2a-1e10406cc354 None admin] RBAC: Adding query filter params (name=service) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:15.472310 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109010b8-714f-472e-bd2a-1e10406cc354 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:15.473646 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109010b8-714f-472e-bd2a-1e10406cc354 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:15.481786 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 329/646] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 12:31:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:15.482588 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109010b8-714f-472e-bd2a-1e10406cc354 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:15.487468 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c24dfa8-91b4-4422-864d-20f2492af276 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:15.488919 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c24dfa8-91b4-4422-864d-20f2492af276 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:15.504016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-2c24dfa8-91b4-4422-864d-20f2492af276 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:15.504016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:15.504016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:15.504016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:15.504016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:15.504016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:15.504016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:15.504016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:15.504016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:15.504016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:15.504016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:15.504016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:15.504016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:15.504016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:15.504016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:15.504016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:15.505023 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:15.505023 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:15.505023 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:15.505023 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:15.505023 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:15.505023 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:15.505023 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:15.505023 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:15.505023 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:15.505023 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:15.505023 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:15.505023 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:15.505023 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:15.505023 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:15.505023 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:15.505023 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:15.505023 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:15.506832 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:15.506832 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:15.506832 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:15.506832 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 318/647] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:15.507065 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c24dfa8-91b4-4422-864d-20f2492af276 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:15.529564 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:15.592682 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109010b8-714f-472e-bd2a-1e10406cc354 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:15.607871 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c24dfa8-91b4-4422-864d-20f2492af276 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:15.617771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:15.627834 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8391182d-d745-4b26-ac61-0eebb6db761d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:15.631168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8391182d-d745-4b26-ac61-0eebb6db761d None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:15.631168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8391182d-d745-4b26-ac61-0eebb6db761d None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:15.631168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8391182d-d745-4b26-ac61-0eebb6db761d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:15.639436 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8391182d-d745-4b26-ac61-0eebb6db761d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:15.640826 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8391182d-d745-4b26-ac61-0eebb6db761d None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:15.644435 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ba093-9db2-433d-96d0-66ca7356f13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:15.645064 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ba093-9db2-433d-96d0-66ca7356f13c None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:15.645342 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ba093-9db2-433d-96d0-66ca7356f13c None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:15.645615 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ba093-9db2-433d-96d0-66ca7356f13c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:15.658478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-8391182d-d745-4b26-ac61-0eebb6db761d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:15.658478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:15.658478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:15.658478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:15.658478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:15.658478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:15.658478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:15.658478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:15.658478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:15.658478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:15.658478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:15.658478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:15.658478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:15.658478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:15.658478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:15.658478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:15.659806 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:15.659806 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:15.659806 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:15.659806 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:15.659806 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:15.659806 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:15.659806 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:15.659806 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:15.659806 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:15.659806 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:15.659806 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:15.659806 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:15.659806 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:15.659806 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:15.659806 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:15.659806 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:15.659806 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:15.660699 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:15.660699 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:15.660699 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:15.660699 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 330/648] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:15.660934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8391182d-d745-4b26-ac61-0eebb6db761d None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:15.663351 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0ba093-9db2-433d-96d0-66ca7356f13c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:15.664429 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0ba093-9db2-433d-96d0-66ca7356f13c None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:15.680748 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-8a0ba093-9db2-433d-96d0-66ca7356f13c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:15.680748 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:15.680748 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:15.680748 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:15.680748 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:15.680748 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:15.680748 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:15.680748 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:15.680748 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:15.680748 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:15.680748 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:15.680748 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:15.680748 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:15.680748 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:15.680748 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:15.680748 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:15.681771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:15.681771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:15.681771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:15.681771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:15.681771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:15.681771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:15.681771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:15.681771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:15.681771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:15.681771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:15.681771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:15.681771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:15.681771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:15.681771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:15.681771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:15.681771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:15.681771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:15.682668 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:15.682668 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:15.682668 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:15.682668 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 319/649] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:15.682668 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ba093-9db2-433d-96d0-66ca7356f13c None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:15.768385 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:15.779618 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8391182d-d745-4b26-ac61-0eebb6db761d None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:15.786365 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ba093-9db2-433d-96d0-66ca7356f13c None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:15.815352 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0fc5e9-da1a-4a74-bda6-068a47078b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:15.815992 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0fc5e9-da1a-4a74-bda6-068a47078b52 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:15.816272 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0fc5e9-da1a-4a74-bda6-068a47078b52 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:15.816545 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0fc5e9-da1a-4a74-bda6-068a47078b52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:15.821349 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54e83b-f007-40cd-a781-f098278c55a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:15.821961 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54e83b-f007-40cd-a781-f098278c55a0 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:15.822238 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54e83b-f007-40cd-a781-f098278c55a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:15.822507 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54e83b-f007-40cd-a781-f098278c55a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:15.823145 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e54e83b-f007-40cd-a781-f098278c55a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:15.823476 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e54e83b-f007-40cd-a781-f098278c55a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:15.824828 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e54e83b-f007-40cd-a781-f098278c55a0 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:15.827598 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0fc5e9-da1a-4a74-bda6-068a47078b52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:15.829022 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0fc5e9-da1a-4a74-bda6-068a47078b52 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:15.837150 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 320/650] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:15.842402 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54e83b-f007-40cd-a781-f098278c55a0 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:15.844524 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-5e0fc5e9-da1a-4a74-bda6-068a47078b52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:15.844524 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:15.844524 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:15.844524 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:15.844524 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:15.844524 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:15.844524 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:15.844524 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:15.844524 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:15.844524 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:15.844524 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:15.844524 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:15.844524 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:15.844524 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:15.844524 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:15.844524 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:15.845425 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:15.845425 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:15.845425 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:15.845425 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:15.845425 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:15.845425 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:15.845425 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:15.845425 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:15.845425 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:15.845425 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:15.845425 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:15.845425 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:15.845425 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:15.845425 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:15.845425 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:15.845425 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:15.845425 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:15.846295 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:15.846295 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:15.846295 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:15.846295 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 331/651] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:15.846513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0fc5e9-da1a-4a74-bda6-068a47078b52 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:15.871855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:15.952488 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54e83b-f007-40cd-a781-f098278c55a0 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:15.955233 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0fc5e9-da1a-4a74-bda6-068a47078b52 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:15.988758 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc613c-b9e0-4fd6-8274-aab4730a499d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:15.989380 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc613c-b9e0-4fd6-8274-aab4730a499d None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:15.989721 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc613c-b9e0-4fd6-8274-aab4730a499d None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:15.990159 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc613c-b9e0-4fd6-8274-aab4730a499d None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:15.991308 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc613c-b9e0-4fd6-8274-aab4730a499d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:15.991645 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc613c-b9e0-4fd6-8274-aab4730a499d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:15.993011 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc613c-b9e0-4fd6-8274-aab4730a499d None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:15.995705 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d7563-33de-4a6b-98c6-175d5b4331c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:15.996508 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d7563-33de-4a6b-98c6-175d5b4331c6 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:15.996959 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d7563-33de-4a6b-98c6-175d5b4331c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:15.997404 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d7563-33de-4a6b-98c6-175d5b4331c6 None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:15.998224 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d7563-33de-4a6b-98c6-175d5b4331c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:15.998718 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d7563-33de-4a6b-98c6-175d5b4331c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:16.000250 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d7563-33de-4a6b-98c6-175d5b4331c6 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:16.005062 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 321/652] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:16.006702 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc613c-b9e0-4fd6-8274-aab4730a499d None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:16.018383 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 332/653] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:16.018383 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d7563-33de-4a6b-98c6-175d5b4331c6 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:16.022777 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:16.099248 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:16.121536 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc613c-b9e0-4fd6-8274-aab4730a499d None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:16.122869 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d7563-33de-4a6b-98c6-175d5b4331c6 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:16.156981 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fcce8e-e622-4f99-8182-2b21227c0237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:16.157616 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fcce8e-e622-4f99-8182-2b21227c0237 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:16.157732 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d0633-4c66-4bc9-81d3-ce9fd0773cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:16.157911 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fcce8e-e622-4f99-8182-2b21227c0237 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:16.158160 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fcce8e-e622-4f99-8182-2b21227c0237 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:16.158346 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d0633-4c66-4bc9-81d3-ce9fd0773cae None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:16.158605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d0633-4c66-4bc9-81d3-ce9fd0773cae None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:16.159047 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d0633-4c66-4bc9-81d3-ce9fd0773cae None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:16.159711 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5d0633-4c66-4bc9-81d3-ce9fd0773cae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:16.160050 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5d0633-4c66-4bc9-81d3-ce9fd0773cae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:16.161367 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5d0633-4c66-4bc9-81d3-ce9fd0773cae None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:16.165149 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fcce8e-e622-4f99-8182-2b21227c0237 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:16.166576 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fcce8e-e622-4f99-8182-2b21227c0237 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:16.178778 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 333/654] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:16.178778 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d0633-4c66-4bc9-81d3-ce9fd0773cae None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:16.179035 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-91fcce8e-e622-4f99-8182-2b21227c0237 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:16.179035 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:16.179035 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:16.179035 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:16.179035 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:16.179035 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:16.179035 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:16.179035 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:16.179035 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:16.179035 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:16.179035 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:16.179035 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:16.179035 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 12:31:16.179035 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 12:31:16.179035 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 12:31:16.179901 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 12:31:16.179901 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:16.179901 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:16.179901 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:16.179901 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:16.179901 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:16.179901 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:16.179901 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:16.179901 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:16.179901 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:16.179901 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:16.179901 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:16.179901 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:16.179901 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:16.179901 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:16.179901 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:16.179901 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:16.180793 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 12:31:16.180793 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 12:31:16.180793 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 12:31:16.180793 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 12:31:16.180793 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 12:31:16.180793 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 12:31:16.180793 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:16.180793 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:16.180793 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 322/655] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:31:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:16.180793 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fcce8e-e622-4f99-8182-2b21227c0237 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:16.255021 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:16.299076 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fcce8e-e622-4f99-8182-2b21227c0237 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:16.300105 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d0633-4c66-4bc9-81d3-ce9fd0773cae None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:16.343247 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ccfbe-0ebc-42db-9ba4-7f04866ba40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:16.344822 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ccfbe-0ebc-42db-9ba4-7f04866ba40e None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:16.345259 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ccfbe-0ebc-42db-9ba4-7f04866ba40e None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:16.345685 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ccfbe-0ebc-42db-9ba4-7f04866ba40e None admin] PATH_INFO: `/v3/users/glance` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:16.348470 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caacee8b-26ca-47ff-bc70-fdadcc848fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:16.349678 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caacee8b-26ca-47ff-bc70-fdadcc848fbb None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:16.349958 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caacee8b-26ca-47ff-bc70-fdadcc848fbb None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:16.350372 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caacee8b-26ca-47ff-bc70-fdadcc848fbb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:16.354843 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ccfbe-0ebc-42db-9ba4-7f04866ba40e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:16.356792 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ccfbe-0ebc-42db-9ba4-7f04866ba40e None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:16.357224 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caacee8b-26ca-47ff-bc70-fdadcc848fbb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:16.358637 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caacee8b-26ca-47ff-bc70-fdadcc848fbb None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:16.360589 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:16.370585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-291ccfbe-0ebc-42db-9ba4-7f04866ba40e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 03 12:31:16.370585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:16.370585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:16.370585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:16.370585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:16.370585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:16.370585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:16.370585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:16.370585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:16.370585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:16.370585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:16.370585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:16.370585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 12:31:16.370585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 12:31:16.370585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 12:31:16.370585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 12:31:16.372439 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:16.372439 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:16.372439 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 12:31:16.372439 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:16.372439 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 12:31:16.372439 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:16.372439 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:16.372439 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:16.372439 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:16.372439 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:16.372439 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:16.372439 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:16.372439 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:16.372439 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:16.372439 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:16.372439 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:16.372439 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:16.372439 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:16.373556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:16.373556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:16.373556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 12:31:16.373556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 12:31:16.373556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 12:31:16.373556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 12:31:16.373556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 12:31:16.373556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 12:31:16.373556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 03 12:31:16.373556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:16.374214 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-caacee8b-26ca-47ff-bc70-fdadcc848fbb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:16.374214 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:16.374214 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:16.374214 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:16.374214 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:16.374214 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:16.374214 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:16.374214 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:16.374214 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:16.374214 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:16.374214 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:16.374214 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:16.374214 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 12:31:16.374214 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 12:31:16.374214 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 12:31:16.375145 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 12:31:16.375145 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:16.375145 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:16.375145 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:16.375145 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:16.375145 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:16.375145 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:16.375145 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:16.375145 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:16.375145 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:16.375145 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:16.375145 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:16.375145 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:16.375145 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:16.375145 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:16.375145 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:16.375145 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:16.376395 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 12:31:16.376395 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 12:31:16.376395 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 12:31:16.376395 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 12:31:16.376395 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 12:31:16.376395 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 12:31:16.376395 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:16.376395 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:16.376948 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 323/656] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:31:16 2022] GET /identity/v3/users/glance => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:16.377128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ccfbe-0ebc-42db-9ba4-7f04866ba40e None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:16.378406 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 334/657] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:31:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:16.379205 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caacee8b-26ca-47ff-bc70-fdadcc848fbb None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:16.475053 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ccfbe-0ebc-42db-9ba4-7f04866ba40e None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:16.475592 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caacee8b-26ca-47ff-bc70-fdadcc848fbb None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:16.496709 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:16.514188 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00179b3b-e1a8-4f5c-b09e-a8422ae361d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:16.514343 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d83567-0743-44eb-974a-ffd8f58cfb44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:16.514447 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00179b3b-e1a8-4f5c-b09e-a8422ae361d4 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:16.514447 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00179b3b-e1a8-4f5c-b09e-a8422ae361d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:16.514634 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d83567-0743-44eb-974a-ffd8f58cfb44 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:16.514719 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00179b3b-e1a8-4f5c-b09e-a8422ae361d4 None admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:16.514799 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d83567-0743-44eb-974a-ffd8f58cfb44 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:16.514799 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d83567-0743-44eb-974a-ffd8f58cfb44 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:16.514952 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00179b3b-e1a8-4f5c-b09e-a8422ae361d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:16.514952 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00179b3b-e1a8-4f5c-b09e-a8422ae361d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:16.514952 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00179b3b-e1a8-4f5c-b09e-a8422ae361d4 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:16.517270 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d83567-0743-44eb-974a-ffd8f58cfb44 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:16.518715 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d83567-0743-44eb-974a-ffd8f58cfb44 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:16.525700 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 335/658] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 12:31:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:16.526946 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3743f-df6c-4b48-9f65-3ded1b2dba6e None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:16.527231 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3743f-df6c-4b48-9f65-3ded1b2dba6e None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:16.527527 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3743f-df6c-4b48-9f65-3ded1b2dba6e None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:16.530965 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 336/659] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:16.530965 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3743f-df6c-4b48-9f65-3ded1b2dba6e None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:16.535491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-d1d83567-0743-44eb-974a-ffd8f58cfb44 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 03 12:31:16.535491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:16.535491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:16.535491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:16.535491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:16.535491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:16.535491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:16.535491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:16.535491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:16.535491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:16.535491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:16.535491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:16.535491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 12:31:16.535491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 12:31:16.535491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 12:31:16.535491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 12:31:16.536605 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:16.536605 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:16.536605 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 12:31:16.536605 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:16.536605 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 12:31:16.536605 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:16.536605 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:16.536605 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:16.536605 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:16.536605 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:16.536605 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:16.536605 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:16.536605 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:16.536605 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:16.536605 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:16.536605 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:16.536605 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:16.536605 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:16.538398 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:16.538398 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:16.538398 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 12:31:16.538398 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 12:31:16.538398 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 12:31:16.538398 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 12:31:16.538398 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 12:31:16.538398 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 12:31:16.538398 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 03 12:31:16.538398 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:16.538398 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 324/660] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 12:31:16 2022] GET /identity/v3/users/neutron => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:16.538398 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d83567-0743-44eb-974a-ffd8f58cfb44 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:16.591126 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:16.634041 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d83567-0743-44eb-974a-ffd8f58cfb44 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:16.644718 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3743f-df6c-4b48-9f65-3ded1b2dba6e None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:16.667882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d2baa0-05e8-4f91-902a-fc1a125190cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:16.668517 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d2baa0-05e8-4f91-902a-fc1a125190cd None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:16.668792 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d2baa0-05e8-4f91-902a-fc1a125190cd None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:16.669070 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d2baa0-05e8-4f91-902a-fc1a125190cd None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:16.669798 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d2baa0-05e8-4f91-902a-fc1a125190cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:16.670096 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d2baa0-05e8-4f91-902a-fc1a125190cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:16.671584 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d2baa0-05e8-4f91-902a-fc1a125190cd None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:16.679718 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4be6ee0-0420-40a1-a1eb-4ddc7bcdd4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:16.680322 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be6ee0-0420-40a1-a1eb-4ddc7bcdd4e8 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:16.680604 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be6ee0-0420-40a1-a1eb-4ddc7bcdd4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:16.680884 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be6ee0-0420-40a1-a1eb-4ddc7bcdd4e8 None admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:16.681602 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4be6ee0-0420-40a1-a1eb-4ddc7bcdd4e8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:16.681913 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4be6ee0-0420-40a1-a1eb-4ddc7bcdd4e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:16.683308 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4be6ee0-0420-40a1-a1eb-4ddc7bcdd4e8 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:16.683903 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 325/661] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 12:31:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:16.684992 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d2baa0-05e8-4f91-902a-fc1a125190cd None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:16.705499 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 337/662] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Oct 3 12:31:16 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:16.706581 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0f27f-39ee-416d-b3c0-c3cd3fcf186c None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:16.706873 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0f27f-39ee-416d-b3c0-c3cd3fcf186c None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:16.707148 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0f27f-39ee-416d-b3c0-c3cd3fcf186c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:16.724958 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-e8f0f27f-39ee-416d-b3c0-c3cd3fcf186c None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:16.726159 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-e8f0f27f-39ee-416d-b3c0-c3cd3fcf186c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:16.760407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:16.786805 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d2baa0-05e8-4f91-902a-fc1a125190cd None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:16.792174 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 338/663] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:31:16.793298 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f0f27f-39ee-416d-b3c0-c3cd3fcf186c None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:16.821297 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:16.838356 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de54b19d-9838-4018-939f-49f64fa6e282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:16.839189 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de54b19d-9838-4018-939f-49f64fa6e282 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:16.839504 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de54b19d-9838-4018-939f-49f64fa6e282 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:16.839778 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de54b19d-9838-4018-939f-49f64fa6e282 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:16.840497 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de54b19d-9838-4018-939f-49f64fa6e282 None admin] RBAC: Adding query filter params (role.id=ff6b2d4413644db39a2bd8576afc6ccb, scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7, user.id=524e9184eee1486cb071cad5c67050c9) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:16.840804 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de54b19d-9838-4018-939f-49f64fa6e282 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:16.842714 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de54b19d-9838-4018-939f-49f64fa6e282 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:16.855358 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 326/664] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:31:16 2022] GET /identity/v3/role_assignments?user.id=524e9184eee1486cb071cad5c67050c9&scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7&role.id=ff6b2d4413644db39a2bd8576afc6ccb => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:16.855358 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de54b19d-9838-4018-939f-49f64fa6e282 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:16.925193 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f0f27f-39ee-416d-b3c0-c3cd3fcf186c None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:16.969660 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e77322-a718-466c-8eca-1bc859e079cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:16.970284 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e77322-a718-466c-8eca-1bc859e079cc None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:16.970560 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e77322-a718-466c-8eca-1bc859e079cc None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:16.970831 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e77322-a718-466c-8eca-1bc859e079cc None admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:16.971612 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e77322-a718-466c-8eca-1bc859e079cc None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:16.971916 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e77322-a718-466c-8eca-1bc859e079cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:16.973552 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e77322-a718-466c-8eca-1bc859e079cc None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:16.999203 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 339/665] 10.222.0.33 () {56 vars in 1139 bytes} [Mon Oct 3 12:31:16 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:16.999926 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de54b19d-9838-4018-939f-49f64fa6e282 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:17.000689 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e77322-a718-466c-8eca-1bc859e079cc None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:17.039916 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40404571-01fe-4e48-b199-ef334e6340da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:17.040504 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40404571-01fe-4e48-b199-ef334e6340da None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:17.040785 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40404571-01fe-4e48-b199-ef334e6340da None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:17.041108 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40404571-01fe-4e48-b199-ef334e6340da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:17.041848 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40404571-01fe-4e48-b199-ef334e6340da None admin] RBAC: Adding query filter params (role.id=ff6b2d4413644db39a2bd8576afc6ccb, scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7, user.id=990f5837994f4a088c656c9b33e9a0dc) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:17.042153 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40404571-01fe-4e48-b199-ef334e6340da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:17.043609 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40404571-01fe-4e48-b199-ef334e6340da None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:17.052144 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 327/666] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:31:16 2022] GET /identity/v3/role_assignments?user.id=990f5837994f4a088c656c9b33e9a0dc&scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7&role.id=ff6b2d4413644db39a2bd8576afc6ccb => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:17.053114 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40404571-01fe-4e48-b199-ef334e6340da None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:17.058667 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:17.119130 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:17.171704 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e77322-a718-466c-8eca-1bc859e079cc None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:17.187053 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40404571-01fe-4e48-b199-ef334e6340da None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:17.219629 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae30de2-7efa-4588-8843-9dc374e91761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:17.220575 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae30de2-7efa-4588-8843-9dc374e91761 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:17.221113 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae30de2-7efa-4588-8843-9dc374e91761 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:17.221585 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae30de2-7efa-4588-8843-9dc374e91761 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:17.228063 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f7412-b8c4-49bf-8614-4266dd32ea82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:17.228713 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f7412-b8c4-49bf-8614-4266dd32ea82 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:17.228993 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f7412-b8c4-49bf-8614-4266dd32ea82 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:17.229280 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f7412-b8c4-49bf-8614-4266dd32ea82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:17.314534 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f7412-b8c4-49bf-8614-4266dd32ea82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:17.315871 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f7412-b8c4-49bf-8614-4266dd32ea82 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:17.322187 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae30de2-7efa-4588-8843-9dc374e91761 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:17.324019 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae30de2-7efa-4588-8843-9dc374e91761 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:17.364259 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:17.395679 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:17.447854 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 340/667] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:31:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 12:31:17.448630 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 328/668] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:31:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 12:31:17.449345 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae30de2-7efa-4588-8843-9dc374e91761 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:17.454331 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f7412-b8c4-49bf-8614-4266dd32ea82 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:17.571569 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f7412-b8c4-49bf-8614-4266dd32ea82 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:17.575943 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae30de2-7efa-4588-8843-9dc374e91761 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:17.608245 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:17.614187 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdceb240-5ff6-4a61-98a2-3edcce45ef81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:17.621920 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdceb240-5ff6-4a61-98a2-3edcce45ef81 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:17.621920 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdceb240-5ff6-4a61-98a2-3edcce45ef81 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:17.621920 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdceb240-5ff6-4a61-98a2-3edcce45ef81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:17.622188 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a97ae-5f34-4bd3-8644-08c190a92452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:17.622188 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a97ae-5f34-4bd3-8644-08c190a92452 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:17.622188 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a97ae-5f34-4bd3-8644-08c190a92452 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:17.622188 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a97ae-5f34-4bd3-8644-08c190a92452 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:17.627603 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdceb240-5ff6-4a61-98a2-3edcce45ef81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:17.629215 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdceb240-5ff6-4a61-98a2-3edcce45ef81 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:17.634167 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672a97ae-5f34-4bd3-8644-08c190a92452 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:31:17.639191 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:17.640380 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-672a97ae-5f34-4bd3-8644-08c190a92452 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:31:17.640380 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:17.640380 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:17.640380 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:17.640380 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:17.640380 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:17.640380 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:17.640380 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:17.640380 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:17.640380 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:17.640380 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:17.640380 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:17.640380 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:31:17.640380 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:31:17.640380 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:31:17.640380 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:31:17.641486 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:31:17.641486 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:31:17.641486 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:31:17.641486 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:31:17.641486 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:17.641486 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:17.641486 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:17.641486 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:17.641486 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:17.641486 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:17.641486 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:17.641486 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:17.641486 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:17.641486 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:17.641486 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:17.641486 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:17.642511 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:17.642511 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:17.642511 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:17.642511 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:17.642511 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:31:17.642511 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:31:17.642511 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:31:17.642511 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:31:17.642511 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:31:17.642511 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:31:17.642511 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:31:17.642511 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:17.647528 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 341/669] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 12:31:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:17.647528 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a97ae-5f34-4bd3-8644-08c190a92452 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:17.656592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-fdceb240-5ff6-4a61-98a2-3edcce45ef81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:17.656592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:17.656592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:17.656592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:17.656592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:17.656592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:17.656592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:17.656592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:17.656592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:17.656592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:17.656592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:17.656592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:17.656592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:17.656592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:17.656592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:17.656592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:17.657776 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:17.657776 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:17.657776 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:17.657776 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:17.657776 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:17.657776 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:17.657776 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:17.657776 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:17.657776 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:17.657776 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:17.657776 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:17.657776 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:17.657776 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:17.657776 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:17.657776 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:17.657776 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:17.657776 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:17.658869 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:17.658869 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:17.658869 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:17.660903 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 329/670] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:17.662488 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdceb240-5ff6-4a61-98a2-3edcce45ef81 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:17.773460 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a97ae-5f34-4bd3-8644-08c190a92452 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:17.785030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdceb240-5ff6-4a61-98a2-3edcce45ef81 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:17.810272 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0b989-8b8f-49e3-923a-a0a39edf3fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:17.811163 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0b989-8b8f-49e3-923a-a0a39edf3fe6 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:17.811636 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0b989-8b8f-49e3-923a-a0a39edf3fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:17.812095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0b989-8b8f-49e3-923a-a0a39edf3fe6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:17.822097 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc07afd-8454-4978-911c-02e4e775e88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:17.822405 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc07afd-8454-4978-911c-02e4e775e88d None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:17.822688 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc07afd-8454-4978-911c-02e4e775e88d None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:17.822961 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc07afd-8454-4978-911c-02e4e775e88d None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:17.823735 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc07afd-8454-4978-911c-02e4e775e88d None admin] RBAC: Adding query filter params (name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:17.824079 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc07afd-8454-4978-911c-02e4e775e88d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:17.825222 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd0b989-8b8f-49e3-923a-a0a39edf3fe6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:17.825406 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc07afd-8454-4978-911c-02e4e775e88d None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:17.827113 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd0b989-8b8f-49e3-923a-a0a39edf3fe6 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:17.839651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 330/671] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 12:31:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:17.840542 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc07afd-8454-4978-911c-02e4e775e88d None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:17.844127 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-6dd0b989-8b8f-49e3-923a-a0a39edf3fe6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:17.844127 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:17.844127 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:17.844127 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:17.844127 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:17.844127 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:17.844127 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:17.844127 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:17.844127 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:17.844127 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:17.844127 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:17.844127 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:17.844127 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:17.844127 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:17.844127 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:17.844127 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:17.845047 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:17.845047 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:17.845047 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:17.845047 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:17.845047 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:17.845047 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:17.845047 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:17.845047 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:17.845047 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:17.845047 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:17.845047 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:17.845047 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:17.845047 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:17.845047 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:17.845047 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:17.845047 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:17.845047 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:17.845988 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:17.845988 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:17.845988 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:17.847804 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 342/672] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:17.848742 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0b989-8b8f-49e3-923a-a0a39edf3fe6 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:17.897506 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:17.914166 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:17.955700 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc07afd-8454-4978-911c-02e4e775e88d None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:17.960871 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0b989-8b8f-49e3-923a-a0a39edf3fe6 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:17.993819 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e1d7ab-c5fc-40fb-bc8e-c65dfe9e417c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:17.994706 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1d7ab-c5fc-40fb-bc8e-c65dfe9e417c None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:17.995207 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1d7ab-c5fc-40fb-bc8e-c65dfe9e417c None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:17.995750 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1d7ab-c5fc-40fb-bc8e-c65dfe9e417c None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:17.996625 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e1d7ab-c5fc-40fb-bc8e-c65dfe9e417c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:17.997090 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e1d7ab-c5fc-40fb-bc8e-c65dfe9e417c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:17.997955 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb7b2c-3312-468b-be70-e90a749f36a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:17.998586 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb7b2c-3312-468b-be70-e90a749f36a3 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:17.998865 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb7b2c-3312-468b-be70-e90a749f36a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:17.999134 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb7b2c-3312-468b-be70-e90a749f36a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:17.999279 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e1d7ab-c5fc-40fb-bc8e-c65dfe9e417c None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:18.010769 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb7b2c-3312-468b-be70-e90a749f36a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:18.012165 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb7b2c-3312-468b-be70-e90a749f36a3 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:18.013411 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 331/673] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:18.014577 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e1d7ab-c5fc-40fb-bc8e-c65dfe9e417c None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:18.028030 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-e1fb7b2c-3312-468b-be70-e90a749f36a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:18.028030 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:18.028030 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:18.028030 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:18.028030 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:18.028030 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:18.028030 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:18.028030 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:18.028030 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:18.028030 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:18.028030 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:18.028030 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:18.028030 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:18.028030 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:18.028030 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:18.028030 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:18.028992 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:18.028992 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:18.028992 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:18.028992 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:18.028992 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:18.028992 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:18.028992 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:18.028992 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:18.028992 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:18.028992 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:18.028992 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:18.028992 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:18.028992 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:18.028992 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:18.028992 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:18.028992 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:18.028992 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:18.029888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:18.029888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:18.029888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:18.033958 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 343/674] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:18.034868 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb7b2c-3312-468b-be70-e90a749f36a3 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:18.125614 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e1d7ab-c5fc-40fb-bc8e-c65dfe9e417c None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:18.135791 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:18.144850 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb7b2c-3312-468b-be70-e90a749f36a3 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:18.151768 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:18.164087 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b3ac8-32f7-437e-8b39-d0abfd2a11a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:18.164958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b3ac8-32f7-437e-8b39-d0abfd2a11a7 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:18.165406 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b3ac8-32f7-437e-8b39-d0abfd2a11a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:18.165823 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b3ac8-32f7-437e-8b39-d0abfd2a11a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:18.166594 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4b3ac8-32f7-437e-8b39-d0abfd2a11a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:18.167040 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4b3ac8-32f7-437e-8b39-d0abfd2a11a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:18.168599 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4b3ac8-32f7-437e-8b39-d0abfd2a11a7 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:18.185084 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f44085-4613-4720-bbc7-3a45b12d5d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:18.185084 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f44085-4613-4720-bbc7-3a45b12d5d75 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:18.185084 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f44085-4613-4720-bbc7-3a45b12d5d75 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:18.185084 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f44085-4613-4720-bbc7-3a45b12d5d75 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:18.185456 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 332/675] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:18.185456 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b3ac8-32f7-437e-8b39-d0abfd2a11a7 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:18.192854 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f44085-4613-4720-bbc7-3a45b12d5d75 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:18.195434 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f44085-4613-4720-bbc7-3a45b12d5d75 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:18.206912 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-e4f44085-4613-4720-bbc7-3a45b12d5d75 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:18.206912 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:18.206912 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:18.206912 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:18.206912 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:18.206912 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:18.206912 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:18.206912 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:18.206912 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:18.206912 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:18.206912 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:18.206912 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:18.206912 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 12:31:18.206912 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 12:31:18.206912 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 12:31:18.208020 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 12:31:18.208020 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:18.208020 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:18.208020 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:18.208020 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:18.208020 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:18.208020 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:18.208020 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:18.208020 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:18.208020 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:18.208020 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:18.208020 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:18.208020 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:18.208020 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:18.208020 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:18.208020 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:18.208020 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:18.209745 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 12:31:18.209745 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 12:31:18.209745 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 12:31:18.209745 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 12:31:18.209745 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 12:31:18.209745 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 12:31:18.209745 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:18.209745 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:18.209745 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 344/676] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:31:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:18.210252 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f44085-4613-4720-bbc7-3a45b12d5d75 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:18.297638 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b3ac8-32f7-437e-8b39-d0abfd2a11a7 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:18.320045 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f44085-4613-4720-bbc7-3a45b12d5d75 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:18.342957 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18fa395-5328-4f2a-98f8-5262e75aa579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:18.343608 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18fa395-5328-4f2a-98f8-5262e75aa579 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:18.344165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18fa395-5328-4f2a-98f8-5262e75aa579 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:18.344434 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18fa395-5328-4f2a-98f8-5262e75aa579 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:18.345066 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18fa395-5328-4f2a-98f8-5262e75aa579 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:18.345518 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18fa395-5328-4f2a-98f8-5262e75aa579 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:18.347489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18fa395-5328-4f2a-98f8-5262e75aa579 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:18.357088 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6652f3-49f3-4080-94f3-3ce050b90c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:18.357801 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6652f3-49f3-4080-94f3-3ce050b90c68 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:18.358220 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6652f3-49f3-4080-94f3-3ce050b90c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:18.358577 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6652f3-49f3-4080-94f3-3ce050b90c68 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:18.367831 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 333/677] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:18.368543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6652f3-49f3-4080-94f3-3ce050b90c68 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:18.368727 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18fa395-5328-4f2a-98f8-5262e75aa579 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:18.370225 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6652f3-49f3-4080-94f3-3ce050b90c68 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:18.384839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-bb6652f3-49f3-4080-94f3-3ce050b90c68 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:18.384839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:18.384839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:18.384839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:18.384839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:18.384839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:18.384839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:18.384839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:18.384839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:18.384839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:18.384839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:18.384839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:18.384839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 12:31:18.384839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 12:31:18.384839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 12:31:18.385744 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 12:31:18.385744 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:18.385744 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:18.385744 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:18.385744 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:18.385744 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:18.385744 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:18.385744 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:18.385744 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:18.385744 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:18.385744 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:18.385744 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:18.385744 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:18.385744 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:18.385744 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:18.385744 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:18.385744 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:18.386634 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 12:31:18.386634 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 12:31:18.386634 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 12:31:18.386634 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 12:31:18.386634 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 12:31:18.386634 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 12:31:18.386634 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:18.386634 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:18.390799 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 345/678] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:31:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:18.391642 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6652f3-49f3-4080-94f3-3ce050b90c68 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:18.408878 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:18.422074 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:18.485006 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18fa395-5328-4f2a-98f8-5262e75aa579 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:18.507823 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6652f3-49f3-4080-94f3-3ce050b90c68 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:18.522380 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad5b8a1-c640-49cf-b96a-8a6199a78131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:18.523393 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5b8a1-c640-49cf-b96a-8a6199a78131 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:18.523893 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5b8a1-c640-49cf-b96a-8a6199a78131 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:18.524361 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5b8a1-c640-49cf-b96a-8a6199a78131 None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:18.525258 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad5b8a1-c640-49cf-b96a-8a6199a78131 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:18.525773 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad5b8a1-c640-49cf-b96a-8a6199a78131 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:18.527430 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad5b8a1-c640-49cf-b96a-8a6199a78131 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:18.539885 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 334/679] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 12:31:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:18.540964 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad5b8a1-c640-49cf-b96a-8a6199a78131 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:18.549825 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c863cbc1-82ae-44a8-88c3-08de7b930768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:18.551152 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863cbc1-82ae-44a8-88c3-08de7b930768 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:18.551667 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863cbc1-82ae-44a8-88c3-08de7b930768 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:18.552136 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863cbc1-82ae-44a8-88c3-08de7b930768 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:18.561598 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c863cbc1-82ae-44a8-88c3-08de7b930768 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:18.563452 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c863cbc1-82ae-44a8-88c3-08de7b930768 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:18.579452 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-c863cbc1-82ae-44a8-88c3-08de7b930768 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 03 12:31:18.579452 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:18.579452 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:18.579452 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:18.579452 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:18.579452 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:18.579452 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:18.579452 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:18.579452 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:18.579452 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:18.579452 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:18.579452 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:18.579452 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 12:31:18.579452 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 12:31:18.579452 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 12:31:18.579452 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 12:31:18.580410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:18.580410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:18.580410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 12:31:18.580410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:18.580410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 12:31:18.580410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:18.580410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:18.580410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:18.580410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:18.580410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:18.580410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:18.580410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:18.580410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:18.580410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:18.580410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:18.580410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:18.580410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:18.580410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:18.581413 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:18.581413 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:18.581413 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 12:31:18.581413 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 12:31:18.581413 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 12:31:18.581413 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 12:31:18.581413 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 12:31:18.581413 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 12:31:18.581413 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 03 12:31:18.581413 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:18.584299 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 346/680] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 12:31:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:18.585680 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c863cbc1-82ae-44a8-88c3-08de7b930768 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:18.669495 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:18.674312 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad5b8a1-c640-49cf-b96a-8a6199a78131 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:18.675982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:18.704460 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c863cbc1-82ae-44a8-88c3-08de7b930768 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:18.725542 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0c284-8338-4b54-8c2f-22143f327ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:18.726520 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0c284-8338-4b54-8c2f-22143f327ee2 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:18.726815 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0c284-8338-4b54-8c2f-22143f327ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:18.727081 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0c284-8338-4b54-8c2f-22143f327ee2 None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:18.727800 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b0c284-8338-4b54-8c2f-22143f327ee2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:18.728104 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b0c284-8338-4b54-8c2f-22143f327ee2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:18.729589 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b0c284-8338-4b54-8c2f-22143f327ee2 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:18.745909 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 335/681] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 12:31:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:18.746697 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73285344-1ad1-44f2-839f-1ab46fce40ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:18.746822 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0c284-8338-4b54-8c2f-22143f327ee2 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:18.747700 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73285344-1ad1-44f2-839f-1ab46fce40ab None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:18.748177 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73285344-1ad1-44f2-839f-1ab46fce40ab None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:18.748668 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73285344-1ad1-44f2-839f-1ab46fce40ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:18.749688 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73285344-1ad1-44f2-839f-1ab46fce40ab None admin] RBAC: Adding query filter params (role.id=ff6b2d4413644db39a2bd8576afc6ccb, scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7, user.id=b697e4847b13496d98a543675f5df1c4) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:18.750617 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73285344-1ad1-44f2-839f-1ab46fce40ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:18.752287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73285344-1ad1-44f2-839f-1ab46fce40ab None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:18.766374 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 347/682] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:31:18 2022] GET /identity/v3/role_assignments?user.id=b697e4847b13496d98a543675f5df1c4&scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7&role.id=ff6b2d4413644db39a2bd8576afc6ccb => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:18.767851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73285344-1ad1-44f2-839f-1ab46fce40ab None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:18.877067 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0c284-8338-4b54-8c2f-22143f327ee2 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:18.929914 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb3dba-f1b9-4054-a75e-a05feb478460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:18.935599 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb3dba-f1b9-4054-a75e-a05feb478460 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:18.935599 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb3dba-f1b9-4054-a75e-a05feb478460 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:18.935599 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb3dba-f1b9-4054-a75e-a05feb478460 None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:18.935599 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceb3dba-f1b9-4054-a75e-a05feb478460 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:18.935599 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceb3dba-f1b9-4054-a75e-a05feb478460 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:18.935599 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceb3dba-f1b9-4054-a75e-a05feb478460 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:18.956032 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 336/683] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Oct 3 12:31:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:18.968560 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb3dba-f1b9-4054-a75e-a05feb478460 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:19.004871 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73285344-1ad1-44f2-839f-1ab46fce40ab None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:19.011247 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:19.062871 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9737dc76-e815-42e6-82c9-87c0ec3e2b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:19.064004 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9737dc76-e815-42e6-82c9-87c0ec3e2b2e None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:19.064625 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9737dc76-e815-42e6-82c9-87c0ec3e2b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:19.065188 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9737dc76-e815-42e6-82c9-87c0ec3e2b2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:19.066118 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9737dc76-e815-42e6-82c9-87c0ec3e2b2e None admin] RBAC: Adding query filter params (role.id=ff6b2d4413644db39a2bd8576afc6ccb, scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7, user.id=eb327a8a18ca414ab8b4111a4d13aad8) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:19.066624 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9737dc76-e815-42e6-82c9-87c0ec3e2b2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:19.068850 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9737dc76-e815-42e6-82c9-87c0ec3e2b2e None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:19.073038 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:19.080947 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 348/684] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:31:18 2022] GET /identity/v3/role_assignments?user.id=eb327a8a18ca414ab8b4111a4d13aad8&scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7&role.id=ff6b2d4413644db39a2bd8576afc6ccb => generated 608 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:19.118941 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb3dba-f1b9-4054-a75e-a05feb478460 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:19.160001 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ead096-64a9-4f01-a5d8-b335799d5ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:19.160883 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ead096-64a9-4f01-a5d8-b335799d5ee3 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:19.161331 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ead096-64a9-4f01-a5d8-b335799d5ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:19.161997 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ead096-64a9-4f01-a5d8-b335799d5ee3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:19.245402 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ead096-64a9-4f01-a5d8-b335799d5ee3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:19.247112 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ead096-64a9-4f01-a5d8-b335799d5ee3 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:19.270818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:19.276935 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c571e0f-2bef-48b9-839c-0118676971a5 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:19.277219 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c571e0f-2bef-48b9-839c-0118676971a5 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:19.277495 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c571e0f-2bef-48b9-839c-0118676971a5 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:19.278839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 349/685] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:19.284903 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b71121-c628-4c80-92f8-cea8cc8461b3 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:19.285308 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b71121-c628-4c80-92f8-cea8cc8461b3 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:19.285692 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b71121-c628-4c80-92f8-cea8cc8461b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:19.307142 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-d1b71121-c628-4c80-92f8-cea8cc8461b3 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:19.308420 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-d1b71121-c628-4c80-92f8-cea8cc8461b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:19.370156 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 337/686] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:31:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 12:31:19.376529 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ead096-64a9-4f01-a5d8-b335799d5ee3 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:19.406625 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 350/687] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 12:31:19.502828 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ead096-64a9-4f01-a5d8-b335799d5ee3 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:19.514410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff8580-6983-48ed-8e9f-bea169a6a6d5 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:19.514703 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff8580-6983-48ed-8e9f-bea169a6a6d5 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:19.514979 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff8580-6983-48ed-8e9f-bea169a6a6d5 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:19.516600 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 351/688] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:19.522847 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fdf9e5-7202-4b05-8e21-08475d4cc879 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:19.523133 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fdf9e5-7202-4b05-8e21-08475d4cc879 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:19.523436 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fdf9e5-7202-4b05-8e21-08475d4cc879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:19.526130 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:19.539845 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50e785e-cb2c-4ee3-ad8c-a3b4fbe94602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:19.540484 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e785e-cb2c-4ee3-ad8c-a3b4fbe94602 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:19.540760 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e785e-cb2c-4ee3-ad8c-a3b4fbe94602 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:19.541026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e785e-cb2c-4ee3-ad8c-a3b4fbe94602 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:19.553920 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50e785e-cb2c-4ee3-ad8c-a3b4fbe94602 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:19.555435 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50e785e-cb2c-4ee3-ad8c-a3b4fbe94602 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:19.564377 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-50fdf9e5-7202-4b05-8e21-08475d4cc879 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:19.565600 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-50fdf9e5-7202-4b05-8e21-08475d4cc879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:19.573867 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-e50e785e-cb2c-4ee3-ad8c-a3b4fbe94602 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:19.573867 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:19.573867 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:19.573867 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:19.573867 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:19.573867 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:19.573867 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:19.573867 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:19.573867 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:19.573867 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:19.573867 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:19.573867 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:19.573867 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:19.573867 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:19.573867 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:19.573867 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:19.575026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:19.575026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:19.575026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:19.575026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:19.575026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:19.575026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:19.575026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:19.575026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:19.575026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:19.575026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:19.575026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:19.575026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:19.575026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:19.575026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:19.575026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:19.575026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:19.575026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:19.576145 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:19.576145 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:19.576145 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:19.578330 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 338/689] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:31:19.583922 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50e785e-cb2c-4ee3-ad8c-a3b4fbe94602 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:19.663036 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 352/690] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:31:19.714435 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fdf9e5-7202-4b05-8e21-08475d4cc879 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:19.724824 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50e785e-cb2c-4ee3-ad8c-a3b4fbe94602 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:19.770142 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76614a21-173b-4193-9ec4-c38b6be4d945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:19.771000 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76614a21-173b-4193-9ec4-c38b6be4d945 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:19.771821 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76614a21-173b-4193-9ec4-c38b6be4d945 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:19.772236 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76614a21-173b-4193-9ec4-c38b6be4d945 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:19.773093 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76614a21-173b-4193-9ec4-c38b6be4d945 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:19.773537 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76614a21-173b-4193-9ec4-c38b6be4d945 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:19.775079 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76614a21-173b-4193-9ec4-c38b6be4d945 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:19.790204 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 339/691] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:31:19.796415 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76614a21-173b-4193-9ec4-c38b6be4d945 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:19.829267 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:19.842600 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:19.885652 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fdf9e5-7202-4b05-8e21-08475d4cc879 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:19.923991 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db9a55-d8f4-406b-a694-4414a5080e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:19.924871 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db9a55-d8f4-406b-a694-4414a5080e6f None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:19.925315 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db9a55-d8f4-406b-a694-4414a5080e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:19.925741 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db9a55-d8f4-406b-a694-4414a5080e6f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:19.926451 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42db9a55-d8f4-406b-a694-4414a5080e6f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:19.928691 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42db9a55-d8f4-406b-a694-4414a5080e6f None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:19.939652 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-42db9a55-d8f4-406b-a694-4414a5080e6f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 03 12:31:19.939652 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:19.939652 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 12:31:19.939652 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 12:31:19.939652 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 12:31:19.939652 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 12:31:19.939652 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 12:31:19.939652 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:31:19.939652 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 03 12:31:19.939652 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:19.939652 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 12:31:19.939652 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:19.939652 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:19.939652 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:19.939652 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:19.939652 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:19.939652 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:19.940879 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:19.940879 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:19.940879 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:19.940879 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:19.940879 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:19.940879 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:19.940879 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 12:31:19.940879 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 12:31:19.940879 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 12:31:19.940879 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 12:31:19.940879 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:19.940879 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:19.940879 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:19.940879 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:19.940879 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:19.940879 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:19.941924 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:19.941924 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:19.941924 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:19.941924 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:19.941924 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:19.941924 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:19.941924 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:19.941924 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:19.941924 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:19.941924 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:19.941924 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 12:31:19.941924 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:31:19.941924 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 03 12:31:19.941924 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:19.944311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 353/692] 10.222.0.33 () {56 vars in 1129 bytes} [Mon Oct 3 12:31:19 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:31:19.952407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db9a55-d8f4-406b-a694-4414a5080e6f None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:19.992525 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76614a21-173b-4193-9ec4-c38b6be4d945 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:20.048672 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67615e31-fd30-4002-af67-4851035a975b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:20.049460 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67615e31-fd30-4002-af67-4851035a975b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:20.049835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67615e31-fd30-4002-af67-4851035a975b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:20.050201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67615e31-fd30-4002-af67-4851035a975b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:20.057418 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db9a55-d8f4-406b-a694-4414a5080e6f None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:20.059617 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67615e31-fd30-4002-af67-4851035a975b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:20.061984 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67615e31-fd30-4002-af67-4851035a975b None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:20.076618 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-67615e31-fd30-4002-af67-4851035a975b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:20.076618 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:20.076618 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:20.076618 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:20.076618 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:20.076618 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:20.076618 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:20.076618 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:20.076618 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:20.076618 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:20.076618 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:20.076618 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:20.076618 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 12:31:20.076618 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 12:31:20.076618 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 12:31:20.077752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 12:31:20.077752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:20.077752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:20.077752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:20.077752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:20.077752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:20.077752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:20.077752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:20.077752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:20.077752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:20.077752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:20.077752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:20.077752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:20.077752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:20.077752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:20.077752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:20.077752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:20.078675 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 12:31:20.078675 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 12:31:20.078675 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 12:31:20.078675 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 12:31:20.078675 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 12:31:20.078675 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 12:31:20.078675 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:20.078675 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:20.083468 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 340/693] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:31:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:20.086264 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67615e31-fd30-4002-af67-4851035a975b None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:20.095700 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:20.104268 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bc914b-9aa0-4244-95d3-a3734b74b616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:20.104860 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc914b-9aa0-4244-95d3-a3734b74b616 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:20.105189 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc914b-9aa0-4244-95d3-a3734b74b616 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:20.105467 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc914b-9aa0-4244-95d3-a3734b74b616 None admin] PATH_INFO: `/v3/services` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:20.106096 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bc914b-9aa0-4244-95d3-a3734b74b616 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:20.106398 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bc914b-9aa0-4244-95d3-a3734b74b616 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:20.108148 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bc914b-9aa0-4244-95d3-a3734b74b616 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:20.113218 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 354/694] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Oct 3 12:31:19 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:20.123255 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bc914b-9aa0-4244-95d3-a3734b74b616 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:20.194767 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:20.203561 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67615e31-fd30-4002-af67-4851035a975b None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:20.235207 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bc914b-9aa0-4244-95d3-a3734b74b616 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:20.242271 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc7cb93-1637-432d-a7b8-23b54623c0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:20.243067 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7cb93-1637-432d-a7b8-23b54623c0c6 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:20.243486 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7cb93-1637-432d-a7b8-23b54623c0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:20.243879 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7cb93-1637-432d-a7b8-23b54623c0c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:20.256994 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc7cb93-1637-432d-a7b8-23b54623c0c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:31:20.262113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-cdc7cb93-1637-432d-a7b8-23b54623c0c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:31:20.262113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:20.262113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:20.262113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:20.262113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:20.262113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:20.262113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:20.262113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:20.262113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:20.262113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:20.262113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:20.262113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:20.262113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:31:20.262113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:31:20.262113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:31:20.262113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:31:20.263201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:31:20.263201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:31:20.263201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:31:20.263201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:31:20.263201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:20.263201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:20.263201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:20.263201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:20.263201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:20.263201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:20.263201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:20.263201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:20.263201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:20.263201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:20.263201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:20.263201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:20.264261 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:20.264261 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:20.264261 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:20.264261 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:20.264261 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:31:20.264261 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:31:20.264261 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:31:20.264261 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:31:20.264261 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:31:20.264261 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:31:20.264261 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:31:20.264261 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:20.267129 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 341/695] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 12:31:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:20.267250 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc7cb93-1637-432d-a7b8-23b54623c0c6 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:20.274937 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8591cf8-0494-4fa0-b5b3-047b6128bfdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:20.275580 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8591cf8-0494-4fa0-b5b3-047b6128bfdf None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:20.275860 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8591cf8-0494-4fa0-b5b3-047b6128bfdf None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:20.276128 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8591cf8-0494-4fa0-b5b3-047b6128bfdf None admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:20.276812 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8591cf8-0494-4fa0-b5b3-047b6128bfdf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:20.277121 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8591cf8-0494-4fa0-b5b3-047b6128bfdf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:20.278638 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8591cf8-0494-4fa0-b5b3-047b6128bfdf None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:20.293986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 355/696] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 12:31:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:20.294629 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8591cf8-0494-4fa0-b5b3-047b6128bfdf None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:20.349073 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:20.382868 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc7cb93-1637-432d-a7b8-23b54623c0c6 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:20.412487 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8591cf8-0494-4fa0-b5b3-047b6128bfdf None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:20.424807 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d455b53-1646-443e-a4d3-b94c7d3e940c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:20.425781 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d455b53-1646-443e-a4d3-b94c7d3e940c None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:20.426649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d455b53-1646-443e-a4d3-b94c7d3e940c None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:20.428341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d455b53-1646-443e-a4d3-b94c7d3e940c None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:20.429522 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d455b53-1646-443e-a4d3-b94c7d3e940c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:20.430331 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d455b53-1646-443e-a4d3-b94c7d3e940c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:20.432179 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d455b53-1646-443e-a4d3-b94c7d3e940c None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:20.443951 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 342/697] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Oct 3 12:31:20 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:20.445412 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d455b53-1646-443e-a4d3-b94c7d3e940c None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:20.450145 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87d199-87b8-4efd-8149-a69f6d6fb287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:20.450780 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87d199-87b8-4efd-8149-a69f6d6fb287 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:20.451068 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87d199-87b8-4efd-8149-a69f6d6fb287 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:20.451417 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87d199-87b8-4efd-8149-a69f6d6fb287 None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:20.452079 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed87d199-87b8-4efd-8149-a69f6d6fb287 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:20.452378 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed87d199-87b8-4efd-8149-a69f6d6fb287 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:20.453792 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed87d199-87b8-4efd-8149-a69f6d6fb287 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:20.464297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 356/698] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:31:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:20.469856 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87d199-87b8-4efd-8149-a69f6d6fb287 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:20.489217 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:20.603551 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87d199-87b8-4efd-8149-a69f6d6fb287 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:20.648345 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:20.697737 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d455b53-1646-443e-a4d3-b94c7d3e940c None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:20.700654 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359d78b-6d73-478e-8e57-fec51d204447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:20.701489 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359d78b-6d73-478e-8e57-fec51d204447 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:20.701774 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359d78b-6d73-478e-8e57-fec51d204447 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:20.702047 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359d78b-6d73-478e-8e57-fec51d204447 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:20.733470 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6359d78b-6d73-478e-8e57-fec51d204447 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:20.736468 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6359d78b-6d73-478e-8e57-fec51d204447 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:20.754962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-6359d78b-6d73-478e-8e57-fec51d204447 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:20.754962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:20.754962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:20.754962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:20.754962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:20.754962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:20.754962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:20.754962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:20.754962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:20.754962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:20.754962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:20.754962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:20.754962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:20.754962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:20.754962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:20.754962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:20.759777 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:20.759777 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:20.759777 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:20.759777 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:20.759777 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:20.759777 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:20.759777 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:20.759777 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:20.759777 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:20.759777 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:20.759777 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:20.759777 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:20.759777 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:20.759777 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:20.759777 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:20.759777 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:20.759777 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:20.762758 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:20.762758 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:20.762758 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:20.762758 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 357/699] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:20.770349 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359d78b-6d73-478e-8e57-fec51d204447 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:20.800538 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5461cd5d-bf6f-4a3f-8a21-d7a69e78aa03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:20.801164 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461cd5d-bf6f-4a3f-8a21-d7a69e78aa03 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:20.801438 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461cd5d-bf6f-4a3f-8a21-d7a69e78aa03 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:20.801711 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461cd5d-bf6f-4a3f-8a21-d7a69e78aa03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:20.802986 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5461cd5d-bf6f-4a3f-8a21-d7a69e78aa03 None admin] RBAC: Adding query filter params (role.id=ff6b2d4413644db39a2bd8576afc6ccb, scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7, user.id=b111d72a34144073917de4f566a58233) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:20.803421 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5461cd5d-bf6f-4a3f-8a21-d7a69e78aa03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:20.806681 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5461cd5d-bf6f-4a3f-8a21-d7a69e78aa03 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:20.813090 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 343/700] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:31:20 2022] GET /identity/v3/role_assignments?user.id=b111d72a34144073917de4f566a58233&scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7&role.id=ff6b2d4413644db39a2bd8576afc6ccb => generated 608 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:20.940298 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359d78b-6d73-478e-8e57-fec51d204447 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:21.005013 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04416866-8b6d-42a9-95ea-cebcbc20a02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:21.006212 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04416866-8b6d-42a9-95ea-cebcbc20a02a None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:21.006739 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04416866-8b6d-42a9-95ea-cebcbc20a02a None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:21.007038 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04416866-8b6d-42a9-95ea-cebcbc20a02a None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:21.007727 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04416866-8b6d-42a9-95ea-cebcbc20a02a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:21.008038 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04416866-8b6d-42a9-95ea-cebcbc20a02a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:21.010563 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04416866-8b6d-42a9-95ea-cebcbc20a02a None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:21.033132 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 358/701] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:21.040349 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5461cd5d-bf6f-4a3f-8a21-d7a69e78aa03 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:21.200044 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:21.226184 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5461cd5d-bf6f-4a3f-8a21-d7a69e78aa03 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:21.280519 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8240e3-9b9a-4d16-879f-eec463a9f749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:21.281169 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8240e3-9b9a-4d16-879f-eec463a9f749 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:21.281448 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8240e3-9b9a-4d16-879f-eec463a9f749 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:21.281724 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8240e3-9b9a-4d16-879f-eec463a9f749 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:21.295018 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8240e3-9b9a-4d16-879f-eec463a9f749 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:21.297790 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8240e3-9b9a-4d16-879f-eec463a9f749 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:21.312493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-0d8240e3-9b9a-4d16-879f-eec463a9f749 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 03 12:31:21.312493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:21.312493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:21.312493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:21.312493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:21.312493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:21.312493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:21.312493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:21.312493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:21.312493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:21.312493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:21.312493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:21.312493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 12:31:21.312493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 12:31:21.312493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 12:31:21.312493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 12:31:21.314374 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:21.314374 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:21.314374 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 12:31:21.314374 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:21.314374 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 12:31:21.314374 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:21.314374 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:21.314374 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:21.314374 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:21.314374 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:21.314374 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:21.314374 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:21.314374 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:21.314374 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:21.314374 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:21.314374 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:21.314374 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:21.314374 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:21.316133 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:21.316133 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:21.316133 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 12:31:21.316133 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 12:31:21.316133 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 12:31:21.316133 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 12:31:21.316133 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 12:31:21.316133 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 12:31:21.316133 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 03 12:31:21.316133 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:21.316133 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 344/702] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Oct 3 12:31:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:21.322662 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04416866-8b6d-42a9-95ea-cebcbc20a02a None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:21.355756 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:21.467159 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04416866-8b6d-42a9-95ea-cebcbc20a02a None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:21.512825 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9c017-aa04-4fad-b39d-443a051679d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:21.513748 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9c017-aa04-4fad-b39d-443a051679d2 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:21.514213 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9c017-aa04-4fad-b39d-443a051679d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:21.514664 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9c017-aa04-4fad-b39d-443a051679d2 None admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:21.515645 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb9c017-aa04-4fad-b39d-443a051679d2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:21.516164 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb9c017-aa04-4fad-b39d-443a051679d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:21.517765 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb9c017-aa04-4fad-b39d-443a051679d2 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:21.540765 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 359/703] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Oct 3 12:31:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:21.546994 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8240e3-9b9a-4d16-879f-eec463a9f749 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:21.699897 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8240e3-9b9a-4d16-879f-eec463a9f749 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:21.747031 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6513cbe-8c83-4ba4-860d-a6614c683b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:21.747967 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6513cbe-8c83-4ba4-860d-a6614c683b73 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:21.748432 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6513cbe-8c83-4ba4-860d-a6614c683b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:21.748884 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6513cbe-8c83-4ba4-860d-a6614c683b73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:21.753750 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ee59a-3f19-4b8f-8fb6-1dcfb31372a4 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:21.754299 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ee59a-3f19-4b8f-8fb6-1dcfb31372a4 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:21.754299 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ee59a-3f19-4b8f-8fb6-1dcfb31372a4 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:21.756376 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 360/704] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:21.762144 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c280706-99e2-4d45-b7f9-32b8eaeee6f2 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:21.762500 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c280706-99e2-4d45-b7f9-32b8eaeee6f2 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:21.762853 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c280706-99e2-4d45-b7f9-32b8eaeee6f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:21.763661 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:21.793832 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-3c280706-99e2-4d45-b7f9-32b8eaeee6f2 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:21.795025 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-3c280706-99e2-4d45-b7f9-32b8eaeee6f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:21.881001 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 361/705] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:31:21.888261 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6513cbe-8c83-4ba4-860d-a6614c683b73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:21.889980 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6513cbe-8c83-4ba4-860d-a6614c683b73 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:22.002032 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 345/706] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:31:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 12:31:22.003709 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c280706-99e2-4d45-b7f9-32b8eaeee6f2 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:22.019677 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:22.096549 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360e0c3-0a86-4384-81f4-50dcd9651c4c None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:22.097062 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360e0c3-0a86-4384-81f4-50dcd9651c4c None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:22.097174 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360e0c3-0a86-4384-81f4-50dcd9651c4c None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:22.102469 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 346/707] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 12:31:22.103989 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c644e-272d-452c-98ec-af3080b0bf2d None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:22.104272 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c644e-272d-452c-98ec-af3080b0bf2d None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:22.104542 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c644e-272d-452c-98ec-af3080b0bf2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:22.123202 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c280706-99e2-4d45-b7f9-32b8eaeee6f2 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:22.123434 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-579c644e-272d-452c-98ec-af3080b0bf2d None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:22.124534 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-579c644e-272d-452c-98ec-af3080b0bf2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:22.168884 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a66d5-d51d-4ebd-9921-97a21136ba1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:22.169529 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a66d5-d51d-4ebd-9921-97a21136ba1b None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:22.169808 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a66d5-d51d-4ebd-9921-97a21136ba1b None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:22.170076 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a66d5-d51d-4ebd-9921-97a21136ba1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:22.179718 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:22.194018 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258a66d5-d51d-4ebd-9921-97a21136ba1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:22.195495 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258a66d5-d51d-4ebd-9921-97a21136ba1b None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:22.222914 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-258a66d5-d51d-4ebd-9921-97a21136ba1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:22.222914 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:22.222914 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:22.222914 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:22.222914 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:22.222914 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:22.222914 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:22.222914 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:22.222914 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:22.222914 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:22.222914 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:22.222914 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:22.222914 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:22.222914 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:22.222914 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:22.222914 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:22.223949 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:22.223949 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:22.223949 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:22.223949 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:22.223949 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:22.223949 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:22.223949 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:22.223949 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:22.223949 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:22.223949 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:22.223949 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:22.223949 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:22.223949 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:22.223949 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:22.223949 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:22.223949 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:22.223949 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:22.225187 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:22.225187 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:22.225187 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:22.228944 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 362/708] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:22.231192 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a66d5-d51d-4ebd-9921-97a21136ba1b None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:22.258374 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 347/709] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 12:31:22.316444 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c644e-272d-452c-98ec-af3080b0bf2d None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:22.329754 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:22.379919 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a66d5-d51d-4ebd-9921-97a21136ba1b None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:22.448394 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f5f20-84aa-4a1c-9308-25cecf1b4734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:22.449401 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f5f20-84aa-4a1c-9308-25cecf1b4734 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:22.449878 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f5f20-84aa-4a1c-9308-25cecf1b4734 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:22.452735 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f5f20-84aa-4a1c-9308-25cecf1b4734 None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:22.453628 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00f5f20-84aa-4a1c-9308-25cecf1b4734 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:22.454179 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00f5f20-84aa-4a1c-9308-25cecf1b4734 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:22.455782 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00f5f20-84aa-4a1c-9308-25cecf1b4734 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:22.533791 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c644e-272d-452c-98ec-af3080b0bf2d None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:22.593065 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 363/710] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:22.598994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f5f20-84aa-4a1c-9308-25cecf1b4734 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:22.610217 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a0d51-6030-4ea6-832b-51b390a91cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:22.610864 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a0d51-6030-4ea6-832b-51b390a91cf3 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:22.611147 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a0d51-6030-4ea6-832b-51b390a91cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:22.611460 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a0d51-6030-4ea6-832b-51b390a91cf3 None admin] PATH_INFO: `/v3/services/network` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:22.612049 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0a0d51-6030-4ea6-832b-51b390a91cf3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:22.613595 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0a0d51-6030-4ea6-832b-51b390a91cf3 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:22.626587 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-fc0a0d51-6030-4ea6-832b-51b390a91cf3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 03 12:31:22.626587 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:22.626587 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 12:31:22.626587 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 12:31:22.626587 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 12:31:22.626587 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 12:31:22.626587 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 12:31:22.626587 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:31:22.626587 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 03 12:31:22.626587 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:22.626587 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 12:31:22.626587 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:22.626587 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:22.626587 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:22.626587 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:22.626587 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:22.626587 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:22.627850 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:22.627850 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:22.627850 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:22.627850 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:22.627850 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:22.627850 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:22.627850 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 12:31:22.627850 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 12:31:22.627850 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 12:31:22.627850 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 12:31:22.627850 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:22.627850 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:22.627850 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:22.627850 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:22.627850 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:22.627850 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:22.628665 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:22.628665 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:22.628665 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:22.628665 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:22.628665 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:22.628665 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:22.628665 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:22.628665 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:22.628665 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:22.628665 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:22.628665 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 12:31:22.628665 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:31:22.628665 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 03 12:31:22.628665 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:22.628665 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 348/711] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:31:22 2022] GET /identity/v3/services/network => generated 88 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:22.629466 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a0d51-6030-4ea6-832b-51b390a91cf3 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:22.672318 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:22.719604 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f5f20-84aa-4a1c-9308-25cecf1b4734 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:22.742287 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a0d51-6030-4ea6-832b-51b390a91cf3 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:22.747871 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:22.761857 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba16ef-2639-447f-b66a-6fab794f58d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:22.762510 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba16ef-2639-447f-b66a-6fab794f58d0 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:22.762776 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba16ef-2639-447f-b66a-6fab794f58d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:22.763050 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba16ef-2639-447f-b66a-6fab794f58d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:22.772723 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ba16ef-2639-447f-b66a-6fab794f58d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:22.774320 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ba16ef-2639-447f-b66a-6fab794f58d0 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:22.785321 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ae5c4-fe43-46b3-b417-3b4b9c80694c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:22.786256 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ae5c4-fe43-46b3-b417-3b4b9c80694c None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:22.786786 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ae5c4-fe43-46b3-b417-3b4b9c80694c None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:22.787255 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ae5c4-fe43-46b3-b417-3b4b9c80694c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:22.787404 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-72ba16ef-2639-447f-b66a-6fab794f58d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:22.787404 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:22.787404 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:22.787404 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:22.787404 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:22.787404 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:22.787404 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:22.787404 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:22.787404 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:22.787404 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:22.787404 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:22.787404 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:22.787404 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 12:31:22.787404 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 12:31:22.787404 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 12:31:22.788298 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 12:31:22.788298 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:22.788298 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:22.788298 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:22.788298 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:22.788298 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:22.788298 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:22.788298 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:22.788298 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:22.788298 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:22.788298 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:22.788298 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:22.788298 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:22.788298 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:22.788298 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:22.788298 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:22.788298 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:22.789250 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 12:31:22.789250 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 12:31:22.789250 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 12:31:22.789250 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 12:31:22.789250 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 12:31:22.789250 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 12:31:22.789250 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:22.789250 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:22.789250 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 364/712] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:31:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:22.789732 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba16ef-2639-447f-b66a-6fab794f58d0 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:22.803412 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807ae5c4-fe43-46b3-b417-3b4b9c80694c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:22.805342 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807ae5c4-fe43-46b3-b417-3b4b9c80694c None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:22.822232 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-807ae5c4-fe43-46b3-b417-3b4b9c80694c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:22.822232 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:22.822232 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:22.822232 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:22.822232 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:22.822232 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:22.822232 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:22.822232 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:22.822232 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:22.822232 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:22.822232 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:22.822232 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:22.822232 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:22.822232 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:22.822232 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:22.822232 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:22.823218 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:22.823218 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:22.823218 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:22.823218 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:22.823218 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:22.823218 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:22.823218 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:22.823218 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:22.823218 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:22.823218 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:22.823218 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:22.823218 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:22.823218 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:22.823218 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:22.823218 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:22.823218 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:22.823218 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:22.824506 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:22.824506 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:22.824506 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:22.826515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 349/713] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:22.827609 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ae5c4-fe43-46b3-b417-3b4b9c80694c None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:22.921028 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba16ef-2639-447f-b66a-6fab794f58d0 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:22.953113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:22.968968 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ae5c4-fe43-46b3-b417-3b4b9c80694c None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:22.973143 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4cc818-6da3-46d1-85ac-d56904adc111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:22.974110 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4cc818-6da3-46d1-85ac-d56904adc111 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:22.974648 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4cc818-6da3-46d1-85ac-d56904adc111 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:22.975186 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4cc818-6da3-46d1-85ac-d56904adc111 None admin] PATH_INFO: `/v3/services` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:22.976033 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4cc818-6da3-46d1-85ac-d56904adc111 None admin] RBAC: Adding query filter params (name=network) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:22.976977 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4cc818-6da3-46d1-85ac-d56904adc111 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:22.978656 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4cc818-6da3-46d1-85ac-d56904adc111 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:22.984748 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 365/714] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Oct 3 12:31:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:22.985970 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4cc818-6da3-46d1-85ac-d56904adc111 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:23.014023 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2be37c-ea27-487a-a8c8-16fff5e14249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:23.015527 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2be37c-ea27-487a-a8c8-16fff5e14249 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:23.016270 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2be37c-ea27-487a-a8c8-16fff5e14249 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:23.016740 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2be37c-ea27-487a-a8c8-16fff5e14249 None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:23.018274 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2be37c-ea27-487a-a8c8-16fff5e14249 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:23.019121 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2be37c-ea27-487a-a8c8-16fff5e14249 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:23.021451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2be37c-ea27-487a-a8c8-16fff5e14249 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:23.038191 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 350/715] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 12:31:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:23.039624 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2be37c-ea27-487a-a8c8-16fff5e14249 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:23.040617 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:23.110758 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4cc818-6da3-46d1-85ac-d56904adc111 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:23.155280 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cfc612-d076-493b-9c3e-9cf754a0f6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:23.155984 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cfc612-d076-493b-9c3e-9cf754a0f6e6 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:23.156536 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cfc612-d076-493b-9c3e-9cf754a0f6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:23.156669 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cfc612-d076-493b-9c3e-9cf754a0f6e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:23.157274 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cfc612-d076-493b-9c3e-9cf754a0f6e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:23.157585 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cfc612-d076-493b-9c3e-9cf754a0f6e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:23.158989 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cfc612-d076-493b-9c3e-9cf754a0f6e6 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:23.180118 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 366/716] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:23.181104 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cfc612-d076-493b-9c3e-9cf754a0f6e6 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:23.190611 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2be37c-ea27-487a-a8c8-16fff5e14249 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:23.230787 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e3a48-0471-482c-950f-3701ef0be33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:23.231813 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e3a48-0471-482c-950f-3701ef0be33d None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:23.232290 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e3a48-0471-482c-950f-3701ef0be33d None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:23.232774 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e3a48-0471-482c-950f-3701ef0be33d None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:23.233649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063e3a48-0471-482c-950f-3701ef0be33d None admin] RBAC: Adding query filter params (type=network) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:23.234225 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063e3a48-0471-482c-950f-3701ef0be33d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:23.235816 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063e3a48-0471-482c-950f-3701ef0be33d None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:23.241808 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 351/717] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Oct 3 12:31:23 2022] GET /identity/v3/services?type=network => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:23.243037 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e3a48-0471-482c-950f-3701ef0be33d None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:23.265370 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:23.291474 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cfc612-d076-493b-9c3e-9cf754a0f6e6 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:23.300149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:23.332696 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5dbad7-573a-4a8a-91a6-19701b8bde57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:23.333749 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5dbad7-573a-4a8a-91a6-19701b8bde57 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:23.333879 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5dbad7-573a-4a8a-91a6-19701b8bde57 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:23.334192 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5dbad7-573a-4a8a-91a6-19701b8bde57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:23.335001 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5dbad7-573a-4a8a-91a6-19701b8bde57 None admin] RBAC: Adding query filter params (role.id=7933c45dcdd54ef0a98018ba8a8bddaa, scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7, user.id=990f5837994f4a088c656c9b33e9a0dc) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:23.335273 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5dbad7-573a-4a8a-91a6-19701b8bde57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:23.336883 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5dbad7-573a-4a8a-91a6-19701b8bde57 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:23.344728 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 367/718] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:31:23 2022] GET /identity/v3/role_assignments?user.id=990f5837994f4a088c656c9b33e9a0dc&scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7&role.id=7933c45dcdd54ef0a98018ba8a8bddaa => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:23.345534 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e276a2a-580c-4a99-9dd8-d0e9fb1f3564 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:23.346369 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e276a2a-580c-4a99-9dd8-d0e9fb1f3564 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:23.346903 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e276a2a-580c-4a99-9dd8-d0e9fb1f3564 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:23.349769 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 368/719] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:23.361121 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa3f493-e2eb-426d-8b7d-5c0e462f4a57 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:23.361441 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa3f493-e2eb-426d-8b7d-5c0e462f4a57 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:23.361687 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa3f493-e2eb-426d-8b7d-5c0e462f4a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:23.384710 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e3a48-0471-482c-950f-3701ef0be33d None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:23.397820 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-9aa3f493-e2eb-426d-8b7d-5c0e462f4a57 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:23.403178 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-9aa3f493-e2eb-426d-8b7d-5c0e462f4a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:23.473861 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b20d0-30ec-42b6-9754-681b9ea8f5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:23.476205 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b20d0-30ec-42b6-9754-681b9ea8f5f9 None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:23.476925 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b20d0-30ec-42b6-9754-681b9ea8f5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:23.478056 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b20d0-30ec-42b6-9754-681b9ea8f5f9 None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:23.478994 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875b20d0-30ec-42b6-9754-681b9ea8f5f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:23.483077 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875b20d0-30ec-42b6-9754-681b9ea8f5f9 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:23.531559 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 369/720] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:31:23.581903 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 352/721] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Oct 3 12:31:23 2022] POST /identity/v3/users => generated 254 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:31:23.859367 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa3f493-e2eb-426d-8b7d-5c0e462f4a57 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:23.889475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1640418a-cd24-4f5d-ad1d-ccaaa2dc378f None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:23.889792 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1640418a-cd24-4f5d-ad1d-ccaaa2dc378f None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:23.890026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1640418a-cd24-4f5d-ad1d-ccaaa2dc378f None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:23.891895 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 353/722] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 12:31:23.897527 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a284b-0bbe-408f-81d4-7d5f181d8139 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:23.897815 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a284b-0bbe-408f-81d4-7d5f181d8139 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:23.898080 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a284b-0bbe-408f-81d4-7d5f181d8139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:23.928941 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-5f5a284b-0bbe-408f-81d4-7d5f181d8139 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:23.930293 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-5f5a284b-0bbe-408f-81d4-7d5f181d8139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:23.992707 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa3f493-e2eb-426d-8b7d-5c0e462f4a57 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:23.994708 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:24.003842 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:24.012634 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 354/723] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:31:24.034078 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8a4a5-79a8-4d92-aa99-a24f824fd2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:24.035096 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8a4a5-79a8-4d92-aa99-a24f824fd2b1 None admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:24.035580 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8a4a5-79a8-4d92-aa99-a24f824fd2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:24.036004 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8a4a5-79a8-4d92-aa99-a24f824fd2b1 None admin] PATH_INFO: `/v3/services` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:24.038172 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f8a4a5-79a8-4d92-aa99-a24f824fd2b1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:24.039763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f8a4a5-79a8-4d92-aa99-a24f824fd2b1 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:24.155593 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 370/724] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Oct 3 12:31:23 2022] POST /identity/v3/services => generated 251 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:31:24.359949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a284b-0bbe-408f-81d4-7d5f181d8139 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:24.472962 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a284b-0bbe-408f-81d4-7d5f181d8139 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:24.516346 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a9a33f-89d5-4015-b1c1-8714064ead98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:24.516982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9a33f-89d5-4015-b1c1-8714064ead98 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:24.517261 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9a33f-89d5-4015-b1c1-8714064ead98 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:24.517535 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9a33f-89d5-4015-b1c1-8714064ead98 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:24.529724 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a9a33f-89d5-4015-b1c1-8714064ead98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:31:24.534490 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-d6a9a33f-89d5-4015-b1c1-8714064ead98 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:31:24.534490 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:24.534490 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:24.534490 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:24.534490 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:24.534490 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:24.534490 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:24.534490 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:24.534490 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:24.534490 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:24.534490 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:24.534490 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:24.534490 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:31:24.534490 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:31:24.534490 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:31:24.534490 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:31:24.535475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:31:24.535475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:31:24.535475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:31:24.535475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:31:24.535475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:24.535475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:24.535475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:24.535475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:24.535475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:24.535475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:24.535475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:24.535475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:24.535475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:24.535475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:24.535475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:24.535475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:24.536695 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:24.536695 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:24.536695 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:24.536695 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:24.536695 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:31:24.536695 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:31:24.536695 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:31:24.536695 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:31:24.536695 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:31:24.536695 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:31:24.536695 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:31:24.536695 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:24.536695 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 355/725] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 12:31:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:24.542027 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8a4a5-79a8-4d92-aa99-a24f824fd2b1 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:24.649502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8a4a5-79a8-4d92-aa99-a24f824fd2b1 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:24.685700 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1138d1-5c7d-4999-a898-8ea989bfc44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:24.686392 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1138d1-5c7d-4999-a898-8ea989bfc44a None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:24.686679 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1138d1-5c7d-4999-a898-8ea989bfc44a None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:24.686946 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1138d1-5c7d-4999-a898-8ea989bfc44a None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:24.687650 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1138d1-5c7d-4999-a898-8ea989bfc44a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:24.687957 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1138d1-5c7d-4999-a898-8ea989bfc44a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:24.689315 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1138d1-5c7d-4999-a898-8ea989bfc44a None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:24.702782 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 371/726] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:31:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:24.708388 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a9a33f-89d5-4015-b1c1-8714064ead98 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:24.777732 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:24.826855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a9a33f-89d5-4015-b1c1-8714064ead98 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:24.875715 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2e35db-45c4-4ba2-9baa-9dddfeb363e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:24.876354 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e35db-45c4-4ba2-9baa-9dddfeb363e3 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:24.876635 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e35db-45c4-4ba2-9baa-9dddfeb363e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:24.876907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e35db-45c4-4ba2-9baa-9dddfeb363e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:24.888562 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2e35db-45c4-4ba2-9baa-9dddfeb363e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:24.890018 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2e35db-45c4-4ba2-9baa-9dddfeb363e3 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:24.905212 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-bd2e35db-45c4-4ba2-9baa-9dddfeb363e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:24.905212 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:24.905212 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:24.905212 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:24.905212 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:24.905212 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:24.905212 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:24.905212 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:24.905212 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:24.905212 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:24.905212 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:24.905212 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:24.905212 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:24.905212 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:24.905212 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:24.905212 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:24.914301 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:24.914301 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:24.914301 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:24.914301 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:24.914301 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:24.914301 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:24.914301 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:24.914301 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:24.914301 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:24.914301 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:24.914301 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:24.914301 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:24.914301 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:24.914301 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:24.914301 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:24.914301 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:24.914301 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:24.915203 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:24.915203 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:24.915203 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:24.915203 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 356/727] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:24.915451 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1138d1-5c7d-4999-a898-8ea989bfc44a None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:24.941767 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:25.027513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1138d1-5c7d-4999-a898-8ea989bfc44a None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:25.065134 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0397fcdb-7649-465c-abe3-c9f316ac5fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:25.065673 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0397fcdb-7649-465c-abe3-c9f316ac5fb5 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:25.065955 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0397fcdb-7649-465c-abe3-c9f316ac5fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:25.066227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0397fcdb-7649-465c-abe3-c9f316ac5fb5 None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:25.066865 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0397fcdb-7649-465c-abe3-c9f316ac5fb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:25.067166 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0397fcdb-7649-465c-abe3-c9f316ac5fb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:25.069444 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0397fcdb-7649-465c-abe3-c9f316ac5fb5 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:25.084571 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 372/728] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:25.090102 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2e35db-45c4-4ba2-9baa-9dddfeb363e3 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:25.231100 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2e35db-45c4-4ba2-9baa-9dddfeb363e3 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:25.246987 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:25.277303 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b0b483-3ed9-4f87-af97-02ee98884d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:25.277946 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b0b483-3ed9-4f87-af97-02ee98884d66 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:25.278220 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b0b483-3ed9-4f87-af97-02ee98884d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:25.278489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b0b483-3ed9-4f87-af97-02ee98884d66 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:25.292396 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b0b483-3ed9-4f87-af97-02ee98884d66 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:25.294097 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b0b483-3ed9-4f87-af97-02ee98884d66 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:25.311781 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-f2b0b483-3ed9-4f87-af97-02ee98884d66 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 03 12:31:25.311781 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:25.311781 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:25.311781 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:25.311781 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:25.311781 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:25.311781 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:25.311781 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:25.311781 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:25.311781 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:25.311781 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:25.311781 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:25.311781 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 12:31:25.311781 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 12:31:25.311781 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 12:31:25.311781 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 12:31:25.312764 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:25.312764 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:25.312764 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 12:31:25.312764 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:25.312764 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 12:31:25.312764 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:25.312764 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:25.312764 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:25.312764 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:25.312764 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:25.312764 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:25.312764 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:25.312764 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:25.312764 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:25.312764 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:25.312764 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:25.312764 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:25.312764 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:25.314343 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:25.314343 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:25.314343 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 12:31:25.314343 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 12:31:25.314343 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 12:31:25.314343 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 12:31:25.314343 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 12:31:25.314343 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 12:31:25.314343 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 03 12:31:25.314343 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:25.314343 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 357/729] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 12:31:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:25.320485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0397fcdb-7649-465c-abe3-c9f316ac5fb5 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:25.421985 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:25.440311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0397fcdb-7649-465c-abe3-c9f316ac5fb5 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:25.485131 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d9bbb-82cd-4f8c-87d6-228982526c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:25.485745 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d9bbb-82cd-4f8c-87d6-228982526c5a None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:25.486033 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d9bbb-82cd-4f8c-87d6-228982526c5a None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:25.486306 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d9bbb-82cd-4f8c-87d6-228982526c5a None admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:25.487035 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d9bbb-82cd-4f8c-87d6-228982526c5a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:25.487374 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d9bbb-82cd-4f8c-87d6-228982526c5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:25.488856 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d9bbb-82cd-4f8c-87d6-228982526c5a None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:25.513136 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 373/730] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Oct 3 12:31:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:25.518812 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b0b483-3ed9-4f87-af97-02ee98884d66 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:25.643709 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b0b483-3ed9-4f87-af97-02ee98884d66 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:25.680672 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06a109f-8f3e-4f0b-bd20-e6057ae51d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:25.681290 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a109f-8f3e-4f0b-bd20-e6057ae51d28 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:25.681570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a109f-8f3e-4f0b-bd20-e6057ae51d28 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:25.681843 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a109f-8f3e-4f0b-bd20-e6057ae51d28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:25.721336 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:25.775392 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06a109f-8f3e-4f0b-bd20-e6057ae51d28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:25.776847 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06a109f-8f3e-4f0b-bd20-e6057ae51d28 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:25.895422 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 358/731] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:31:25 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 12:31:25.901781 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d9bbb-82cd-4f8c-87d6-228982526c5a None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:26.034086 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d9bbb-82cd-4f8c-87d6-228982526c5a None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:26.076554 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbe7fd-6922-4f95-949c-f36b4a4e6f4c None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:26.077245 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbe7fd-6922-4f95-949c-f36b4a4e6f4c None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:26.077752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbe7fd-6922-4f95-949c-f36b4a4e6f4c None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:26.079744 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 359/732] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:26.086253 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef4d8f4-1002-4cdc-98c0-9d1c91180e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:26.086918 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4d8f4-1002-4cdc-98c0-9d1c91180e36 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:26.087202 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4d8f4-1002-4cdc-98c0-9d1c91180e36 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:26.087518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4d8f4-1002-4cdc-98c0-9d1c91180e36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:26.088524 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae501e0c-b400-4a03-8e46-86b8a2bfec7c None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:26.089005 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae501e0c-b400-4a03-8e46-86b8a2bfec7c None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:26.089469 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae501e0c-b400-4a03-8e46-86b8a2bfec7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:26.105841 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:26.107866 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef4d8f4-1002-4cdc-98c0-9d1c91180e36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:26.109324 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef4d8f4-1002-4cdc-98c0-9d1c91180e36 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:26.110222 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-ae501e0c-b400-4a03-8e46-86b8a2bfec7c None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:26.111580 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ae501e0c-b400-4a03-8e46-86b8a2bfec7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:26.134349 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-4ef4d8f4-1002-4cdc-98c0-9d1c91180e36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:26.134349 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:26.134349 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:26.134349 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:26.134349 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:26.134349 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:26.134349 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:26.134349 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:26.134349 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:26.134349 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:26.134349 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:26.134349 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:26.134349 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:26.134349 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:26.134349 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:26.134349 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:26.135415 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:26.135415 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:26.135415 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:26.135415 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:26.135415 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:26.135415 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:26.135415 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:26.135415 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:26.135415 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:26.135415 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:26.135415 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:26.135415 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:26.135415 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:26.135415 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:26.135415 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:26.135415 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:26.135415 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:26.136403 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:26.136403 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:26.136403 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:26.137485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 374/733] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:26.142344 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef4d8f4-1002-4cdc-98c0-9d1c91180e36 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:26.202198 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:26.212511 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 360/734] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:26 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 12:31:26.213992 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027db15-811d-478f-8784-a99f616d324a None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:26.214270 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027db15-811d-478f-8784-a99f616d324a None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:26.214554 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027db15-811d-478f-8784-a99f616d324a None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:26.216121 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 361/735] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:26.225917 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c863fa-969b-4da4-98f4-09e0d89c570b None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:26.226232 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c863fa-969b-4da4-98f4-09e0d89c570b None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:26.229139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c863fa-969b-4da4-98f4-09e0d89c570b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:26.259570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-67c863fa-969b-4da4-98f4-09e0d89c570b None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:26.260757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-67c863fa-969b-4da4-98f4-09e0d89c570b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:26.313177 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef4d8f4-1002-4cdc-98c0-9d1c91180e36 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:26.378098 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 362/736] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:26 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:31:26.386781 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d9de2-bd7c-4c45-b181-6839ab9ccf4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:26.387424 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d9de2-bd7c-4c45-b181-6839ab9ccf4a None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:26.387710 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d9de2-bd7c-4c45-b181-6839ab9ccf4a None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:26.387984 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d9de2-bd7c-4c45-b181-6839ab9ccf4a None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:26.388632 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d9de2-bd7c-4c45-b181-6839ab9ccf4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:26.388934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d9de2-bd7c-4c45-b181-6839ab9ccf4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:26.390369 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d9de2-bd7c-4c45-b181-6839ab9ccf4a None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:26.407772 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 375/737] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:26.413506 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c863fa-969b-4da4-98f4-09e0d89c570b None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:26.509045 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d9de2-bd7c-4c45-b181-6839ab9ccf4a None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:26.545552 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c863fa-969b-4da4-98f4-09e0d89c570b None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:26.588881 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:26.604007 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:26.608339 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a34487-8183-456f-a2cd-9a43beb17e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:26.608983 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a34487-8183-456f-a2cd-9a43beb17e6f None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:26.612950 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a34487-8183-456f-a2cd-9a43beb17e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:26.612950 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a34487-8183-456f-a2cd-9a43beb17e6f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:26.621915 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a34487-8183-456f-a2cd-9a43beb17e6f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:26.624091 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a34487-8183-456f-a2cd-9a43beb17e6f None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:26.645786 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-99a34487-8183-456f-a2cd-9a43beb17e6f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:26.645786 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:26.645786 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:26.645786 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:26.645786 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:26.645786 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:26.645786 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:26.645786 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:26.645786 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:26.645786 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:26.645786 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:26.645786 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:26.645786 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 12:31:26.645786 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 12:31:26.645786 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 12:31:26.648497 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 12:31:26.648497 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:26.648497 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:26.648497 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:26.648497 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:26.648497 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:26.648497 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:26.648497 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:26.648497 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:26.648497 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:26.648497 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:26.648497 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:26.648497 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:26.648497 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:26.648497 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:26.648497 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:26.648497 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:26.650570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 12:31:26.650570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 12:31:26.650570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 12:31:26.650570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 12:31:26.650570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 12:31:26.650570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 12:31:26.650570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:26.650570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:26.650570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 363/738] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:31:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:31:26.650570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eadf51b-6f17-459b-af16-cf97ef248754 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:26.650570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eadf51b-6f17-459b-af16-cf97ef248754 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:26.656070 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eadf51b-6f17-459b-af16-cf97ef248754 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:26.656212 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 364/739] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:26.656981 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eadf51b-6f17-459b-af16-cf97ef248754 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:26.686505 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d9de2-bd7c-4c45-b181-6839ab9ccf4a None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:26.722176 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8ee12f-605e-4dc1-a9e5-57ad94ebd008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:26.722802 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ee12f-605e-4dc1-a9e5-57ad94ebd008 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:26.723075 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ee12f-605e-4dc1-a9e5-57ad94ebd008 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:26.723370 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ee12f-605e-4dc1-a9e5-57ad94ebd008 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:26.736246 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8ee12f-605e-4dc1-a9e5-57ad94ebd008 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:31:26.744907 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-6e8ee12f-605e-4dc1-a9e5-57ad94ebd008 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:31:26.744907 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:26.744907 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:26.744907 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:26.744907 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:26.744907 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:26.744907 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:26.744907 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:26.744907 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:26.744907 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:26.744907 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:26.744907 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:26.744907 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:31:26.744907 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:31:26.744907 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:31:26.744907 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:31:26.747010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:31:26.747010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:31:26.747010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:31:26.747010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:31:26.747010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:26.747010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:26.747010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:26.747010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:26.747010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:26.747010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:26.747010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:26.747010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:26.747010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:26.747010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:26.747010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:26.747010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:26.748544 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:26.748544 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:26.748544 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:26.748544 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:26.748544 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:31:26.748544 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:31:26.748544 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:31:26.748544 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:31:26.748544 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:31:26.748544 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:31:26.748544 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:31:26.748544 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:26.748544 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 376/740] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 12:31:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:26.748544 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfd845-f291-4d0f-b2b1-0420221f13f8 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:26.749454 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfd845-f291-4d0f-b2b1-0420221f13f8 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:26.749454 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfd845-f291-4d0f-b2b1-0420221f13f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:26.770100 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-00bfd845-f291-4d0f-b2b1-0420221f13f8 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:26.771244 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-00bfd845-f291-4d0f-b2b1-0420221f13f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:26.790401 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eadf51b-6f17-459b-af16-cf97ef248754 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:26.826477 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c9bf6-9174-472b-b143-ff78f5ef4001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:26.827081 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c9bf6-9174-472b-b143-ff78f5ef4001 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:26.827451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c9bf6-9174-472b-b143-ff78f5ef4001 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:26.827728 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c9bf6-9174-472b-b143-ff78f5ef4001 None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:26.828416 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0c9bf6-9174-472b-b143-ff78f5ef4001 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:26.828718 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0c9bf6-9174-472b-b143-ff78f5ef4001 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:26.830139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0c9bf6-9174-472b-b143-ff78f5ef4001 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:26.842017 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 365/741] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 12:31:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:26.843126 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c9bf6-9174-472b-b143-ff78f5ef4001 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:26.843560 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 377/742] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:26 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 12:31:26.844570 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bfd845-f291-4d0f-b2b1-0420221f13f8 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:26.894927 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:26.912979 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:26.969720 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bfd845-f291-4d0f-b2b1-0420221f13f8 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:26.998824 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c9bf6-9174-472b-b143-ff78f5ef4001 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:27.015966 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a03cca-3201-4d1e-9086-5c861425881d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:27.016991 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a03cca-3201-4d1e-9086-5c861425881d None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:27.017497 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a03cca-3201-4d1e-9086-5c861425881d None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:27.017969 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a03cca-3201-4d1e-9086-5c861425881d None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:27.019069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a03cca-3201-4d1e-9086-5c861425881d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:27.019629 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a03cca-3201-4d1e-9086-5c861425881d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:27.022723 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a03cca-3201-4d1e-9086-5c861425881d None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:27.033337 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 378/743] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:31:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:27.034519 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a03cca-3201-4d1e-9086-5c861425881d None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:27.052128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab76e23-92d1-41af-8d38-867ac1a06977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:27.053201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab76e23-92d1-41af-8d38-867ac1a06977 None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:27.053489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab76e23-92d1-41af-8d38-867ac1a06977 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:27.053769 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab76e23-92d1-41af-8d38-867ac1a06977 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:27.054701 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab76e23-92d1-41af-8d38-867ac1a06977 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:27.056109 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab76e23-92d1-41af-8d38-867ac1a06977 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:27.141216 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 366/744] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Oct 3 12:31:26 2022] POST /identity/v3/services => generated 240 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:31:27.142116 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab76e23-92d1-41af-8d38-867ac1a06977 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:27.146590 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a03cca-3201-4d1e-9086-5c861425881d None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:27.150220 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:27.202435 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c223931-9c9d-45ee-9d81-8ea8ebd3d15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:27.203059 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c223931-9c9d-45ee-9d81-8ea8ebd3d15d None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:27.203364 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c223931-9c9d-45ee-9d81-8ea8ebd3d15d None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:27.203643 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c223931-9c9d-45ee-9d81-8ea8ebd3d15d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:27.204393 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c223931-9c9d-45ee-9d81-8ea8ebd3d15d None admin] RBAC: Adding query filter params (role.id=7933c45dcdd54ef0a98018ba8a8bddaa, scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7, user.id=b111d72a34144073917de4f566a58233) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:27.204791 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c223931-9c9d-45ee-9d81-8ea8ebd3d15d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:27.206702 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c223931-9c9d-45ee-9d81-8ea8ebd3d15d None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:27.220660 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 379/745] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:31:27 2022] GET /identity/v3/role_assignments?user.id=b111d72a34144073917de4f566a58233&scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7&role.id=7933c45dcdd54ef0a98018ba8a8bddaa => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:27.220660 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c223931-9c9d-45ee-9d81-8ea8ebd3d15d None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:27.282578 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab76e23-92d1-41af-8d38-867ac1a06977 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:27.286283 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:27.327485 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c490e07-3bd2-4d14-9f6c-b3fbe069c14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:27.328241 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c490e07-3bd2-4d14-9f6c-b3fbe069c14f None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:27.328595 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c490e07-3bd2-4d14-9f6c-b3fbe069c14f None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:27.328943 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c490e07-3bd2-4d14-9f6c-b3fbe069c14f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:27.344115 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c490e07-3bd2-4d14-9f6c-b3fbe069c14f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:27.345881 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c490e07-3bd2-4d14-9f6c-b3fbe069c14f None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:27.365590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-1c490e07-3bd2-4d14-9f6c-b3fbe069c14f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:27.365590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:27.365590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:27.365590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:27.365590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:27.365590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:27.365590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:27.365590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:27.365590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:27.365590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:27.365590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:27.365590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:27.365590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:27.365590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:27.365590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:27.365590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:27.366639 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:27.366639 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:27.366639 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:27.366639 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:27.366639 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:27.366639 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:27.366639 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:27.366639 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:27.366639 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:27.366639 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:27.366639 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:27.366639 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:27.366639 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:27.366639 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:27.366639 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:27.366639 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:27.366639 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:27.367719 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:27.367719 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:27.367719 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:27.368429 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 367/746] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:27.369836 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26c3b0-1ceb-41a5-9cd1-cbbd71d32b2c None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:27.370122 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26c3b0-1ceb-41a5-9cd1-cbbd71d32b2c None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:27.370401 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26c3b0-1ceb-41a5-9cd1-cbbd71d32b2c None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:27.372055 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 368/747] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:27.380149 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91ca72-18e0-498f-8537-6fdfbb7f3bc2 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:27.380613 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91ca72-18e0-498f-8537-6fdfbb7f3bc2 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:27.381610 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91ca72-18e0-498f-8537-6fdfbb7f3bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:27.386766 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c223931-9c9d-45ee-9d81-8ea8ebd3d15d None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:27.404297 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-2e91ca72-18e0-498f-8537-6fdfbb7f3bc2 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:27.405749 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-2e91ca72-18e0-498f-8537-6fdfbb7f3bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:27.444555 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6999dff5-866c-4477-8313-f36f6555a8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:27.445194 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999dff5-866c-4477-8313-f36f6555a8d3 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:27.445472 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999dff5-866c-4477-8313-f36f6555a8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:27.445736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999dff5-866c-4477-8313-f36f6555a8d3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:27.462218 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6999dff5-866c-4477-8313-f36f6555a8d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:31:27.467010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-6999dff5-866c-4477-8313-f36f6555a8d3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:31:27.467010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:27.467010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:27.467010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:27.467010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:27.467010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:27.467010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:27.467010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:27.467010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:27.467010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:27.467010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:27.467010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:27.467010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:31:27.467010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:31:27.467010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:31:27.467010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:31:27.468229 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:31:27.468229 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:31:27.468229 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:31:27.468229 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:31:27.468229 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:27.468229 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:27.468229 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:27.468229 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:27.468229 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:27.468229 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:27.468229 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:27.468229 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:27.468229 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:27.468229 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:27.468229 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:27.468229 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:27.469161 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:27.469161 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:27.469161 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:27.469161 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:27.469161 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:31:27.469161 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:31:27.469161 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:31:27.469161 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:31:27.469161 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:31:27.469161 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:31:27.469161 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:31:27.469161 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:27.469161 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 380/748] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 12:31:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:27.469853 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6999dff5-866c-4477-8313-f36f6555a8d3 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:27.501405 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:27.510289 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 369/749] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:31:27.511558 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91ca72-18e0-498f-8537-6fdfbb7f3bc2 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:27.625670 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:27.645108 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91ca72-18e0-498f-8537-6fdfbb7f3bc2 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:27.674529 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6999dff5-866c-4477-8313-f36f6555a8d3 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:27.755330 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460f3c74-02f3-4c97-ba44-fe97b4876656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:27.756859 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f3c74-02f3-4c97-ba44-fe97b4876656 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:27.756859 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f3c74-02f3-4c97-ba44-fe97b4876656 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:27.757062 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f3c74-02f3-4c97-ba44-fe97b4876656 None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:27.757708 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460f3c74-02f3-4c97-ba44-fe97b4876656 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:27.758017 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460f3c74-02f3-4c97-ba44-fe97b4876656 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:27.763560 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460f3c74-02f3-4c97-ba44-fe97b4876656 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:27.792033 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 381/750] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:27.795205 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460f3c74-02f3-4c97-ba44-fe97b4876656 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:27.842443 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38b2554-f3cd-4c2a-9d4f-961a7b501105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:27.843863 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b2554-f3cd-4c2a-9d4f-961a7b501105 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:27.845577 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b2554-f3cd-4c2a-9d4f-961a7b501105 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:27.846088 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b2554-f3cd-4c2a-9d4f-961a7b501105 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:27.847675 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38b2554-f3cd-4c2a-9d4f-961a7b501105 None admin] RBAC: Adding query filter params (name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:27.848870 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38b2554-f3cd-4c2a-9d4f-961a7b501105 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:27.851720 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38b2554-f3cd-4c2a-9d4f-961a7b501105 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:27.873372 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 370/751] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 12:31:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:27.874593 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38b2554-f3cd-4c2a-9d4f-961a7b501105 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:27.934990 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:27.954165 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460f3c74-02f3-4c97-ba44-fe97b4876656 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:27.999027 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38b2554-f3cd-4c2a-9d4f-961a7b501105 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:27.999421 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9631d7e-99c7-485b-92c3-e73542114698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:28.000131 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9631d7e-99c7-485b-92c3-e73542114698 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:28.000494 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9631d7e-99c7-485b-92c3-e73542114698 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:28.001017 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9631d7e-99c7-485b-92c3-e73542114698 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:28.011590 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9631d7e-99c7-485b-92c3-e73542114698 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:28.013477 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9631d7e-99c7-485b-92c3-e73542114698 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:28.028587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-a9631d7e-99c7-485b-92c3-e73542114698 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 03 12:31:28.028587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:28.028587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:28.028587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:28.028587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:28.028587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:28.028587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:28.028587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:28.028587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:28.028587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:28.028587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:28.028587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:28.028587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 12:31:28.028587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 12:31:28.028587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 12:31:28.028587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 12:31:28.029582 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:28.029582 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:28.029582 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 12:31:28.029582 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:28.029582 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 12:31:28.029582 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:28.029582 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:28.029582 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:28.029582 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:28.029582 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:28.029582 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:28.029582 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:28.029582 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:28.029582 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:28.029582 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:28.029582 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:28.029582 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:28.029582 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:28.030547 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:28.030547 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:28.030547 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 12:31:28.030547 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 12:31:28.030547 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 12:31:28.030547 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 12:31:28.030547 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 12:31:28.030547 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 12:31:28.030547 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 03 12:31:28.030547 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:28.030547 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 382/752] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Oct 3 12:31:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:31:28.031146 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9631d7e-99c7-485b-92c3-e73542114698 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:28.047816 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c540f9a5-b141-454c-8916-bb2f763e2d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:28.048849 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c540f9a5-b141-454c-8916-bb2f763e2d7d None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:28.049396 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c540f9a5-b141-454c-8916-bb2f763e2d7d None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:28.049666 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c540f9a5-b141-454c-8916-bb2f763e2d7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:28.064094 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c540f9a5-b141-454c-8916-bb2f763e2d7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:28.066223 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c540f9a5-b141-454c-8916-bb2f763e2d7d None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:28.081543 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:28.090747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-c540f9a5-b141-454c-8916-bb2f763e2d7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:28.090747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:28.090747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:28.090747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:28.090747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:28.090747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:28.090747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:28.090747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:28.090747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:28.090747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:28.090747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:28.090747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:28.090747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:28.090747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:28.090747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:28.090747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:28.096029 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:28.096029 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:28.096029 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:28.096029 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:28.096029 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:28.096029 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:28.096029 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:28.096029 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:28.096029 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:28.096029 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:28.096029 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:28.096029 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:28.096029 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:28.096029 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:28.096029 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:28.096029 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:28.096029 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:28.096933 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:28.096933 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:28.096933 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:28.098025 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 371/753] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:28.099021 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c540f9a5-b141-454c-8916-bb2f763e2d7d None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:28.159797 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9631d7e-99c7-485b-92c3-e73542114698 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:28.205845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734583d2-2eb5-4b91-a343-8b1bb2bdf059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:28.206413 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734583d2-2eb5-4b91-a343-8b1bb2bdf059 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:28.206691 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734583d2-2eb5-4b91-a343-8b1bb2bdf059 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:28.206958 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734583d2-2eb5-4b91-a343-8b1bb2bdf059 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:28.207610 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734583d2-2eb5-4b91-a343-8b1bb2bdf059 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:28.210533 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734583d2-2eb5-4b91-a343-8b1bb2bdf059 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:28.222235 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-734583d2-2eb5-4b91-a343-8b1bb2bdf059 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 03 12:31:28.222235 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:28.222235 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 12:31:28.222235 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 12:31:28.222235 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 12:31:28.222235 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 12:31:28.222235 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 12:31:28.222235 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:31:28.222235 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 03 12:31:28.222235 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:28.222235 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 12:31:28.222235 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:28.222235 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:28.222235 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:28.222235 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:28.222235 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:28.222235 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:28.223297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:28.223297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:28.223297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:28.223297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:28.223297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:28.223297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:28.223297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 12:31:28.223297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 12:31:28.223297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 12:31:28.223297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 12:31:28.223297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:28.223297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:28.223297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:28.223297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:28.223297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:28.223297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:28.224568 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:28.224568 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:28.224568 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:28.224568 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:28.224568 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:28.224568 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:28.224568 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:28.224568 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:28.224568 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:28.224568 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:28.224568 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 12:31:28.224568 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:31:28.224568 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 03 12:31:28.224568 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:28.228521 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 383/754] 10.222.0.33 () {56 vars in 1129 bytes} [Mon Oct 3 12:31:28 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:28.229392 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734583d2-2eb5-4b91-a343-8b1bb2bdf059 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:28.231094 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c540f9a5-b141-454c-8916-bb2f763e2d7d None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:28.244195 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:28.279308 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b640643-19c2-4569-a598-e7d87abb4524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:28.281245 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b640643-19c2-4569-a598-e7d87abb4524 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:28.281736 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b640643-19c2-4569-a598-e7d87abb4524 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:28.282829 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b640643-19c2-4569-a598-e7d87abb4524 None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:28.284026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b640643-19c2-4569-a598-e7d87abb4524 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:28.284526 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b640643-19c2-4569-a598-e7d87abb4524 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:28.286950 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b640643-19c2-4569-a598-e7d87abb4524 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:28.311655 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 372/755] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Oct 3 12:31:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:28.312731 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b640643-19c2-4569-a598-e7d87abb4524 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:28.344080 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734583d2-2eb5-4b91-a343-8b1bb2bdf059 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:28.383992 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a339258-34a0-45cb-9d25-459554946731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:28.385248 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a339258-34a0-45cb-9d25-459554946731 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:28.385605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a339258-34a0-45cb-9d25-459554946731 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:28.385952 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a339258-34a0-45cb-9d25-459554946731 None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:28.386977 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a339258-34a0-45cb-9d25-459554946731 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:28.387384 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a339258-34a0-45cb-9d25-459554946731 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:28.389004 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a339258-34a0-45cb-9d25-459554946731 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:28.392471 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:28.409103 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 384/756] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:28.409103 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a339258-34a0-45cb-9d25-459554946731 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:28.452148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b640643-19c2-4569-a598-e7d87abb4524 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:28.488002 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:28.503797 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0800e459-9d90-4def-9e72-9e79c1b3ae93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:28.504413 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800e459-9d90-4def-9e72-9e79c1b3ae93 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:28.504686 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800e459-9d90-4def-9e72-9e79c1b3ae93 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:28.504952 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800e459-9d90-4def-9e72-9e79c1b3ae93 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:28.506072 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0800e459-9d90-4def-9e72-9e79c1b3ae93 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:28.506369 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0800e459-9d90-4def-9e72-9e79c1b3ae93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:28.507747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0800e459-9d90-4def-9e72-9e79c1b3ae93 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:28.512536 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 373/757] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Oct 3 12:31:28 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:28.513098 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0800e459-9d90-4def-9e72-9e79c1b3ae93 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:28.539631 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a339258-34a0-45cb-9d25-459554946731 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:28.578152 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f47442f-a21b-49ad-8906-0d15c7263ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:28.579479 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47442f-a21b-49ad-8906-0d15c7263ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:28.580026 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47442f-a21b-49ad-8906-0d15c7263ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:28.580570 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47442f-a21b-49ad-8906-0d15c7263ff0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:28.593615 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f47442f-a21b-49ad-8906-0d15c7263ff0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:28.595900 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f47442f-a21b-49ad-8906-0d15c7263ff0 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:28.613211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-4f47442f-a21b-49ad-8906-0d15c7263ff0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:28.613211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:28.613211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:28.613211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:28.613211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:28.613211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:28.613211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:28.613211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:28.613211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:28.613211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:28.613211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:28.613211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:28.613211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:28.613211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:28.613211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:28.613211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:28.614407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:28.614407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:28.614407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:28.614407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:28.614407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:28.614407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:28.614407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:28.614407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:28.614407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:28.614407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:28.614407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:28.614407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:28.614407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:28.614407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:28.614407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:28.614407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:28.614407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:28.615521 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:28.615521 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:28.615521 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:28.621382 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 385/758] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:28.621382 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f47442f-a21b-49ad-8906-0d15c7263ff0 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:28.648973 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0800e459-9d90-4def-9e72-9e79c1b3ae93 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:28.683919 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:28.697832 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce34da-43b2-48e4-a2a5-422ed2d09de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:28.698479 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce34da-43b2-48e4-a2a5-422ed2d09de0 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:28.698759 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce34da-43b2-48e4-a2a5-422ed2d09de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:28.699033 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce34da-43b2-48e4-a2a5-422ed2d09de0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:28.707731 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce34da-43b2-48e4-a2a5-422ed2d09de0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:28.709725 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce34da-43b2-48e4-a2a5-422ed2d09de0 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:28.726679 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-4cce34da-43b2-48e4-a2a5-422ed2d09de0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 12:31:28.726679 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:28.726679 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:28.726679 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:28.726679 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:28.726679 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:28.726679 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:28.726679 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:28.726679 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:28.726679 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:28.726679 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:28.726679 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:28.726679 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 12:31:28.726679 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 12:31:28.726679 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 12:31:28.726679 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 12:31:28.727661 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:28.727661 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:28.727661 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 12:31:28.727661 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:28.727661 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 12:31:28.727661 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:28.727661 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:28.727661 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:28.727661 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:28.727661 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:28.727661 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:28.727661 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:28.727661 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:28.727661 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:28.727661 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:28.727661 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:28.727661 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:28.727661 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:28.729218 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:28.729218 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:28.729218 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 12:31:28.729218 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 12:31:28.729218 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 12:31:28.729218 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 12:31:28.729218 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 12:31:28.729218 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 12:31:28.729218 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 12:31:28.729218 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:28.729218 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 374/759] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Oct 3 12:31:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:28.729218 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce34da-43b2-48e4-a2a5-422ed2d09de0 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:28.743341 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f47442f-a21b-49ad-8906-0d15c7263ff0 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:28.766479 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:28.782493 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d446d-c3c0-479c-9298-092c0de84055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:28.783130 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d446d-c3c0-479c-9298-092c0de84055 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:28.783453 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d446d-c3c0-479c-9298-092c0de84055 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:28.783728 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d446d-c3c0-479c-9298-092c0de84055 None admin] PATH_INFO: `/v3/services` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:28.784353 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d446d-c3c0-479c-9298-092c0de84055 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:28.784670 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d446d-c3c0-479c-9298-092c0de84055 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:28.786050 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d446d-c3c0-479c-9298-092c0de84055 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:28.795382 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 386/760] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Oct 3 12:31:28 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:28.797319 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d446d-c3c0-479c-9298-092c0de84055 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:28.871987 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce34da-43b2-48e4-a2a5-422ed2d09de0 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:28.915539 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f706f-ecb4-4dc0-b1f2-d54207a9ffce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:28.917021 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f706f-ecb4-4dc0-b1f2-d54207a9ffce None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:28.917585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f706f-ecb4-4dc0-b1f2-d54207a9ffce None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:28.918107 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f706f-ecb4-4dc0-b1f2-d54207a9ffce None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:28.918914 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041f706f-ecb4-4dc0-b1f2-d54207a9ffce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:28.919383 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041f706f-ecb4-4dc0-b1f2-d54207a9ffce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:28.919844 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d446d-c3c0-479c-9298-092c0de84055 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:28.921174 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041f706f-ecb4-4dc0-b1f2-d54207a9ffce None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:28.938428 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 375/761] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:28.939805 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f706f-ecb4-4dc0-b1f2-d54207a9ffce None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:28.961889 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5279a-d4c4-4d9e-a523-49667df4edff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:28.962654 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5279a-d4c4-4d9e-a523-49667df4edff None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:28.962994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5279a-d4c4-4d9e-a523-49667df4edff None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:28.963388 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5279a-d4c4-4d9e-a523-49667df4edff None admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:28.964207 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5279a-d4c4-4d9e-a523-49667df4edff None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:28.965068 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5279a-d4c4-4d9e-a523-49667df4edff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:28.968724 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5279a-d4c4-4d9e-a523-49667df4edff None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:28.994916 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 387/762] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Oct 3 12:31:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:28.994916 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5279a-d4c4-4d9e-a523-49667df4edff None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:29.018479 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:29.048598 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:29.082258 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f706f-ecb4-4dc0-b1f2-d54207a9ffce None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:29.112513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5279a-d4c4-4d9e-a523-49667df4edff None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:29.121002 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a747f1-f66f-4e44-813f-b946a38cc0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:29.121864 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a747f1-f66f-4e44-813f-b946a38cc0ee None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:29.122219 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a747f1-f66f-4e44-813f-b946a38cc0ee None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:29.122866 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a747f1-f66f-4e44-813f-b946a38cc0ee None admin] PATH_INFO: `/v3/endpoints` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:29.123432 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a747f1-f66f-4e44-813f-b946a38cc0ee None admin] RBAC: Adding query filter params (interface=public, service_id=8c35fcc77f8e4207a37accc82e58dd4c, region_id=RegionOne) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:29.123955 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a747f1-f66f-4e44-813f-b946a38cc0ee None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:29.125513 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a747f1-f66f-4e44-813f-b946a38cc0ee None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:29.138754 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 376/763] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Oct 3 12:31:28 2022] GET /identity/v3/endpoints?service_id=8c35fcc77f8e4207a37accc82e58dd4c&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:29.139891 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a747f1-f66f-4e44-813f-b946a38cc0ee None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:29.154356 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f42e027-9eae-43ca-9e76-7f52cf72815a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:29.155575 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42e027-9eae-43ca-9e76-7f52cf72815a None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:29.156042 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42e027-9eae-43ca-9e76-7f52cf72815a None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:29.156517 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42e027-9eae-43ca-9e76-7f52cf72815a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:29.164702 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f42e027-9eae-43ca-9e76-7f52cf72815a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:29.167630 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f42e027-9eae-43ca-9e76-7f52cf72815a None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:29.181337 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-3f42e027-9eae-43ca-9e76-7f52cf72815a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:29.181337 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:29.181337 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:29.181337 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:29.181337 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:29.181337 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:29.181337 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:29.181337 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:29.181337 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:29.181337 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:29.181337 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:29.181337 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:29.181337 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 12:31:29.181337 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 12:31:29.181337 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 12:31:29.182293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 12:31:29.182293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:29.182293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:29.182293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:29.182293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:29.182293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:29.182293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:29.182293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:29.182293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:29.182293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:29.182293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:29.182293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:29.182293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:29.182293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:29.182293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:29.182293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:29.182293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:29.183175 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 12:31:29.183175 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 12:31:29.183175 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 12:31:29.183175 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 12:31:29.183175 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 12:31:29.183175 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 12:31:29.183175 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:29.183175 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:29.191442 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 388/764] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:31:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:29.191442 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f42e027-9eae-43ca-9e76-7f52cf72815a None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:29.257043 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a747f1-f66f-4e44-813f-b946a38cc0ee None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:29.265330 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:29.296754 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ebadf-70fd-43a1-9c50-f6a533cb2bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:29.297391 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ebadf-70fd-43a1-9c50-f6a533cb2bbc None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:29.297665 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ebadf-70fd-43a1-9c50-f6a533cb2bbc None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:29.297934 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ebadf-70fd-43a1-9c50-f6a533cb2bbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:29.321188 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f42e027-9eae-43ca-9e76-7f52cf72815a None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:29.323217 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:29.362654 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529991ad-e109-4b9f-bdad-a24a418bd21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:29.363932 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529991ad-e109-4b9f-bdad-a24a418bd21a None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:29.364518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529991ad-e109-4b9f-bdad-a24a418bd21a None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:29.365550 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529991ad-e109-4b9f-bdad-a24a418bd21a None admin] PATH_INFO: `/v3/services` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:29.366407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529991ad-e109-4b9f-bdad-a24a418bd21a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:29.368263 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529991ad-e109-4b9f-bdad-a24a418bd21a None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:29.375007 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 389/765] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Oct 3 12:31:29 2022] GET /identity/v3/services => generated 772 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:29.376206 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529991ad-e109-4b9f-bdad-a24a418bd21a None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:29.381261 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ebadf-70fd-43a1-9c50-f6a533cb2bbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:29.382664 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ebadf-70fd-43a1-9c50-f6a533cb2bbc None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:29.518449 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529991ad-e109-4b9f-bdad-a24a418bd21a None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:29.565154 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26e6fe-ede5-4201-8df2-1b94d2601919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:29.565792 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26e6fe-ede5-4201-8df2-1b94d2601919 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:29.566071 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26e6fe-ede5-4201-8df2-1b94d2601919 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:29.566341 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26e6fe-ede5-4201-8df2-1b94d2601919 None admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:29.567060 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a26e6fe-ede5-4201-8df2-1b94d2601919 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:29.567368 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a26e6fe-ede5-4201-8df2-1b94d2601919 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:29.569589 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a26e6fe-ede5-4201-8df2-1b94d2601919 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:29.580409 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 377/766] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:31:29 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 12:31:29.585512 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6832a-45b7-4c1c-a2df-2cbb81b0b372 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:29.585818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6832a-45b7-4c1c-a2df-2cbb81b0b372 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:29.586062 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6832a-45b7-4c1c-a2df-2cbb81b0b372 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:29.587425 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 378/767] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:29.588491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417ca68-3449-43c8-b9ce-2bfac4321bd1 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:29.588763 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417ca68-3449-43c8-b9ce-2bfac4321bd1 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:29.589027 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417ca68-3449-43c8-b9ce-2bfac4321bd1 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:29.589944 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 390/768] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 12:31:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:29.591006 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26e6fe-ede5-4201-8df2-1b94d2601919 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:29.595029 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 379/769] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:29.596473 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea2df2-78be-42bc-8f37-73b8f899dbbb None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:29.596940 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea2df2-78be-42bc-8f37-73b8f899dbbb None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:29.597047 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea2df2-78be-42bc-8f37-73b8f899dbbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:29.610290 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:29.617219 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-d9ea2df2-78be-42bc-8f37-73b8f899dbbb None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:29.618491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d9ea2df2-78be-42bc-8f37-73b8f899dbbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:29.626287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:29.693256 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 380/770] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:31:29.695115 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9661a7-5cec-4279-bbf9-b547636725c3 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:29.695613 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9661a7-5cec-4279-bbf9-b547636725c3 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:29.695880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9661a7-5cec-4279-bbf9-b547636725c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:29.714055 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-9e9661a7-5cec-4279-bbf9-b547636725c3 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:29.715229 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-9e9661a7-5cec-4279-bbf9-b547636725c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:29.715669 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26e6fe-ede5-4201-8df2-1b94d2601919 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:29.759418 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089317e1-4291-4502-a9e0-0811dbd24602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:29.760076 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089317e1-4291-4502-a9e0-0811dbd24602 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:29.760355 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089317e1-4291-4502-a9e0-0811dbd24602 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:29.760626 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089317e1-4291-4502-a9e0-0811dbd24602 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:29.770884 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089317e1-4291-4502-a9e0-0811dbd24602 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:29.772928 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089317e1-4291-4502-a9e0-0811dbd24602 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:29.787851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-089317e1-4291-4502-a9e0-0811dbd24602 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:29.787851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:29.787851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:29.787851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:29.787851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:29.787851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:29.787851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:29.787851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:29.787851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:29.787851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:29.787851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:29.787851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:29.787851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:29.787851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:29.787851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:29.787851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:29.788822 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:29.788822 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:29.788822 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:29.788822 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:29.788822 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:29.788822 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:29.788822 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:29.788822 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:29.788822 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:29.788822 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:29.788822 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:29.788822 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:29.788822 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:29.788822 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:29.788822 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:29.788822 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:29.788822 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:29.789730 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:29.789730 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:29.789730 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:29.798278 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 391/771] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:29.798278 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089317e1-4291-4502-a9e0-0811dbd24602 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:29.798508 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 381/772] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:31:29.798508 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9661a7-5cec-4279-bbf9-b547636725c3 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:29.907559 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:29.933420 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:29.961540 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089317e1-4291-4502-a9e0-0811dbd24602 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:29.985424 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9661a7-5cec-4279-bbf9-b547636725c3 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:30.003699 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a6362-d25b-4b64-82f1-f0b8b4c7c19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:30.004519 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a6362-d25b-4b64-82f1-f0b8b4c7c19a None admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:30.004810 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a6362-d25b-4b64-82f1-f0b8b4c7c19a None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:30.005090 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a6362-d25b-4b64-82f1-f0b8b4c7c19a None admin] PATH_INFO: `/v3/projects/4ea2fa8ad01240f7859182ebbe3051c7/users/990f5837994f4a088c656c9b33e9a0dc/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:30.007669 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a6362-d25b-4b64-82f1-f0b8b4c7c19a None admin] RBAC: Authorizing `identity:create_grant(project_id=4ea2fa8ad01240f7859182ebbe3051c7, user_id=990f5837994f4a088c656c9b33e9a0dc, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:30.009215 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a6362-d25b-4b64-82f1-f0b8b4c7c19a None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:30.023882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58004bc8-5915-443a-a53a-d25ae27f0884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:30.023882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58004bc8-5915-443a-a53a-d25ae27f0884 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:30.023882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58004bc8-5915-443a-a53a-d25ae27f0884 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:30.023882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58004bc8-5915-443a-a53a-d25ae27f0884 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:30.024402 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58004bc8-5915-443a-a53a-d25ae27f0884 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:30.024740 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58004bc8-5915-443a-a53a-d25ae27f0884 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:30.026120 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58004bc8-5915-443a-a53a-d25ae27f0884 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:30.038593 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 382/773] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:30.039195 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58004bc8-5915-443a-a53a-d25ae27f0884 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:30.080192 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 392/774] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 12:31:29 2022] PUT /identity/v3/projects/4ea2fa8ad01240f7859182ebbe3051c7/users/990f5837994f4a088c656c9b33e9a0dc/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:31:30.082085 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a6362-d25b-4b64-82f1-f0b8b4c7c19a None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:30.183641 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58004bc8-5915-443a-a53a-d25ae27f0884 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:30.212752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:30.223514 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ae7331-a06e-4779-b661-0694053eae24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:30.225193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae7331-a06e-4779-b661-0694053eae24 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:30.225538 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a6362-d25b-4b64-82f1-f0b8b4c7c19a None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:30.226148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae7331-a06e-4779-b661-0694053eae24 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:30.226607 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae7331-a06e-4779-b661-0694053eae24 None admin] PATH_INFO: `/v3/services/network` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:30.229768 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ae7331-a06e-4779-b661-0694053eae24 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:30.229768 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ae7331-a06e-4779-b661-0694053eae24 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:30.244563 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-15ae7331-a06e-4779-b661-0694053eae24 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 03 12:31:30.244563 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:30.244563 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 12:31:30.244563 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 12:31:30.244563 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 12:31:30.244563 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 12:31:30.244563 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 12:31:30.244563 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:31:30.244563 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 03 12:31:30.244563 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:30.244563 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 12:31:30.244563 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:30.244563 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:30.244563 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:30.244563 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:30.244563 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:30.244563 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:30.245646 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:30.245646 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:30.245646 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:30.245646 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:30.245646 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:30.245646 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:30.245646 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 12:31:30.245646 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 12:31:30.245646 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 12:31:30.245646 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 12:31:30.245646 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:30.245646 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:30.245646 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:30.245646 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:30.245646 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:30.245646 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:30.246491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:30.246491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:30.246491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:30.246491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:30.246491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:30.246491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:30.246491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:30.246491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:30.246491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:30.246491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:30.246491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 12:31:30.246491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:31:30.246491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 03 12:31:30.246491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:30.250930 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 383/775] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:31:30 2022] GET /identity/v3/services/network => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:30.251958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ae7331-a06e-4779-b661-0694053eae24 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:30.280681 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a3846d-98a0-435a-8e25-11cad2e9c6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:30.281574 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3846d-98a0-435a-8e25-11cad2e9c6e6 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:30.282153 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3846d-98a0-435a-8e25-11cad2e9c6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:30.282898 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3846d-98a0-435a-8e25-11cad2e9c6e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:30.292097 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a3846d-98a0-435a-8e25-11cad2e9c6e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:30.293552 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a3846d-98a0-435a-8e25-11cad2e9c6e6 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:30.295491 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:30.309897 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-f7a3846d-98a0-435a-8e25-11cad2e9c6e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:30.309897 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:30.309897 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:30.309897 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:30.309897 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:30.309897 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:30.309897 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:30.309897 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:30.309897 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:30.309897 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:30.309897 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:30.309897 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:30.309897 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 12:31:30.309897 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 12:31:30.309897 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 12:31:30.310842 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 12:31:30.310842 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:30.310842 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:30.310842 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:30.310842 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:30.310842 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:30.310842 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:30.310842 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:30.310842 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:30.310842 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:30.310842 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:30.310842 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:30.310842 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:30.310842 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:30.310842 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:30.310842 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:30.310842 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:30.312563 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 12:31:30.312563 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 12:31:30.312563 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 12:31:30.312563 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 12:31:30.312563 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 12:31:30.312563 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 12:31:30.312563 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:30.312563 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:30.312563 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 393/776] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:31:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:30.312563 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a3846d-98a0-435a-8e25-11cad2e9c6e6 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:30.398154 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ae7331-a06e-4779-b661-0694053eae24 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:30.440469 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a3846d-98a0-435a-8e25-11cad2e9c6e6 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:30.447103 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1db0424-f366-4441-bb6d-ec35f30eba53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:30.448038 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db0424-f366-4441-bb6d-ec35f30eba53 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:30.448499 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db0424-f366-4441-bb6d-ec35f30eba53 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:30.449536 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db0424-f366-4441-bb6d-ec35f30eba53 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:30.464952 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1db0424-f366-4441-bb6d-ec35f30eba53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:31:30.473581 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-b1db0424-f366-4441-bb6d-ec35f30eba53 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:31:30.473581 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:30.473581 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:30.473581 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:30.473581 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:30.473581 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:30.473581 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:30.473581 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:30.473581 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:30.473581 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:30.473581 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:30.473581 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:30.473581 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:31:30.473581 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:31:30.473581 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:31:30.473581 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:31:30.474821 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:31:30.474821 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:31:30.474821 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:31:30.474821 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:31:30.474821 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:30.474821 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:30.474821 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:30.474821 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:30.474821 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:30.474821 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:30.474821 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:30.474821 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:30.474821 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:30.474821 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:30.474821 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:30.474821 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:30.475876 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:30.475876 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:30.475876 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:30.475876 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:30.475876 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:31:30.475876 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:31:30.475876 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:31:30.475876 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:31:30.475876 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:31:30.475876 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:31:30.475876 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:31:30.475876 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:30.478802 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 384/777] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 12:31:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:30.479829 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1db0424-f366-4441-bb6d-ec35f30eba53 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:30.486500 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce167642-c2a9-42f0-b542-a0fd46a682ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:30.487230 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce167642-c2a9-42f0-b542-a0fd46a682ad None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:30.487636 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce167642-c2a9-42f0-b542-a0fd46a682ad None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:30.487998 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce167642-c2a9-42f0-b542-a0fd46a682ad None admin] PATH_INFO: `/v3/services` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:30.488723 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce167642-c2a9-42f0-b542-a0fd46a682ad None admin] RBAC: Adding query filter params (name=network) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:30.489126 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce167642-c2a9-42f0-b542-a0fd46a682ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:30.491125 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce167642-c2a9-42f0-b542-a0fd46a682ad None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:30.499401 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 394/778] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Oct 3 12:31:30 2022] GET /identity/v3/services?name=network => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:30.500541 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce167642-c2a9-42f0-b542-a0fd46a682ad None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:30.551680 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:30.573773 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:30.619931 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce167642-c2a9-42f0-b542-a0fd46a682ad None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:30.620398 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1db0424-f366-4441-bb6d-ec35f30eba53 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:30.660431 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df687644-6d8b-499b-9ef6-626c5897d2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:30.661195 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df687644-6d8b-499b-9ef6-626c5897d2a9 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:30.661860 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df687644-6d8b-499b-9ef6-626c5897d2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:30.662650 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df687644-6d8b-499b-9ef6-626c5897d2a9 None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:30.663644 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df687644-6d8b-499b-9ef6-626c5897d2a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:30.664044 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df687644-6d8b-499b-9ef6-626c5897d2a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:30.665562 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df687644-6d8b-499b-9ef6-626c5897d2a9 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:30.667992 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337ed73-afbe-4567-b80a-a5677ad403c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:30.668826 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337ed73-afbe-4567-b80a-a5677ad403c3 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:30.669255 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337ed73-afbe-4567-b80a-a5677ad403c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:30.669693 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337ed73-afbe-4567-b80a-a5677ad403c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:30.670515 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337ed73-afbe-4567-b80a-a5677ad403c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:30.671014 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337ed73-afbe-4567-b80a-a5677ad403c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:30.672565 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337ed73-afbe-4567-b80a-a5677ad403c3 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:30.686458 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 385/779] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 12:31:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:30.686458 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df687644-6d8b-499b-9ef6-626c5897d2a9 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:30.687056 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 395/780] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:31:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:30.688134 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337ed73-afbe-4567-b80a-a5677ad403c3 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:30.791691 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df687644-6d8b-499b-9ef6-626c5897d2a9 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:30.802891 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:30.814969 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337ed73-afbe-4567-b80a-a5677ad403c3 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:30.827611 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf5df0-14c6-48e3-a40a-d1b57b9dac91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:30.828235 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf5df0-14c6-48e3-a40a-d1b57b9dac91 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:30.828512 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf5df0-14c6-48e3-a40a-d1b57b9dac91 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:30.828783 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf5df0-14c6-48e3-a40a-d1b57b9dac91 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:30.829404 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bf5df0-14c6-48e3-a40a-d1b57b9dac91 None admin] RBAC: Adding query filter params (type=network) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:30.829703 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bf5df0-14c6-48e3-a40a-d1b57b9dac91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:30.831066 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bf5df0-14c6-48e3-a40a-d1b57b9dac91 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:30.836325 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 386/781] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Oct 3 12:31:30 2022] GET /identity/v3/services?type=network => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:30.837228 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf5df0-14c6-48e3-a40a-d1b57b9dac91 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:30.845788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:30.859842 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1531599c-521e-4105-b646-b33f4af853cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:30.860467 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531599c-521e-4105-b646-b33f4af853cc None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:30.860762 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531599c-521e-4105-b646-b33f4af853cc None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:30.861593 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531599c-521e-4105-b646-b33f4af853cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:30.862662 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1531599c-521e-4105-b646-b33f4af853cc None admin] RBAC: Adding query filter params (role.id=ff6b2d4413644db39a2bd8576afc6ccb, scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7, user.id=2e6e4387a7c24bbc82c22eefe2a3d138) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:30.862985 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1531599c-521e-4105-b646-b33f4af853cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:30.864624 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1531599c-521e-4105-b646-b33f4af853cc None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:30.871579 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 396/782] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:31:30 2022] GET /identity/v3/role_assignments?user.id=2e6e4387a7c24bbc82c22eefe2a3d138&scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7&role.id=ff6b2d4413644db39a2bd8576afc6ccb => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:30.872481 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1531599c-521e-4105-b646-b33f4af853cc None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:30.953372 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf5df0-14c6-48e3-a40a-d1b57b9dac91 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:31.018760 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1531599c-521e-4105-b646-b33f4af853cc None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:31.041336 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd30478-153c-4619-b3f5-a880ad73faf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:31.042665 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd30478-153c-4619-b3f5-a880ad73faf5 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:31.042943 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd30478-153c-4619-b3f5-a880ad73faf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:31.043656 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd30478-153c-4619-b3f5-a880ad73faf5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:31.060379 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd30478-153c-4619-b3f5-a880ad73faf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:31.061802 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd30478-153c-4619-b3f5-a880ad73faf5 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:31.094377 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3fafe7-3238-4e15-b028-d2366f3bab5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:31.095000 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fafe7-3238-4e15-b028-d2366f3bab5d None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:31.095282 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fafe7-3238-4e15-b028-d2366f3bab5d None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:31.095585 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fafe7-3238-4e15-b028-d2366f3bab5d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:31.095918 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-6fd30478-153c-4619-b3f5-a880ad73faf5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:31.095918 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:31.095918 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:31.095918 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:31.095918 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:31.095918 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:31.095918 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:31.095918 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:31.095918 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:31.095918 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:31.095918 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:31.095918 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:31.095918 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:31.095918 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:31.095918 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:31.095918 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:31.096810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:31.096810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:31.096810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:31.096810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:31.096810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:31.096810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:31.096810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:31.096810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:31.096810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:31.096810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:31.096810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:31.096810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:31.096810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:31.096810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:31.096810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:31.096810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:31.096810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:31.097668 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:31.097668 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:31.097668 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:31.097842 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3fafe7-3238-4e15-b028-d2366f3bab5d None admin] RBAC: Adding query filter params (interface=public, service_id=f3f8268d0dea49d282088d64a87ce53c, region_id=RegionOne) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:31.097842 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3fafe7-3238-4e15-b028-d2366f3bab5d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:31.097976 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3fafe7-3238-4e15-b028-d2366f3bab5d None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:31.107340 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 387/783] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:31.107511 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 397/784] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Oct 3 12:31:30 2022] GET /identity/v3/endpoints?service_id=f3f8268d0dea49d282088d64a87ce53c&interface=public®ion_id=RegionOne => generated 194 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:31.107819 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3fafe7-3238-4e15-b028-d2366f3bab5d None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:31.114013 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd30478-153c-4619-b3f5-a880ad73faf5 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:31.152647 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:31.170438 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:31.278366 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3fafe7-3238-4e15-b028-d2366f3bab5d None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:31.283888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd30478-153c-4619-b3f5-a880ad73faf5 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:31.352953 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170c1751-22a4-4d0a-89fe-3ba7450d2dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:31.353496 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c1751-22a4-4d0a-89fe-3ba7450d2dbc None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:31.353779 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c1751-22a4-4d0a-89fe-3ba7450d2dbc None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:31.354055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c1751-22a4-4d0a-89fe-3ba7450d2dbc None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:31.354719 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170c1751-22a4-4d0a-89fe-3ba7450d2dbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:31.355024 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170c1751-22a4-4d0a-89fe-3ba7450d2dbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:31.356418 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170c1751-22a4-4d0a-89fe-3ba7450d2dbc None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:31.358544 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ce36c-86aa-4fc2-b47c-545199ca2578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:31.359157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ce36c-86aa-4fc2-b47c-545199ca2578 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:31.359461 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ce36c-86aa-4fc2-b47c-545199ca2578 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:31.359729 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ce36c-86aa-4fc2-b47c-545199ca2578 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:31.360327 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ce36c-86aa-4fc2-b47c-545199ca2578 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:31.361675 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ce36c-86aa-4fc2-b47c-545199ca2578 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:31.374435 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 388/785] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Oct 3 12:31:31 2022] GET /identity/v3/services => generated 772 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:31.390034 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 398/786] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:31.395896 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ce36c-86aa-4fc2-b47c-545199ca2578 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:31.534713 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:31.551004 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ce36c-86aa-4fc2-b47c-545199ca2578 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:31.588437 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762dc81e-ff5d-49f8-b7de-6876a5adb8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:31.589075 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762dc81e-ff5d-49f8-b7de-6876a5adb8fe None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:31.589347 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762dc81e-ff5d-49f8-b7de-6876a5adb8fe None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:31.589614 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762dc81e-ff5d-49f8-b7de-6876a5adb8fe None admin] PATH_INFO: `/v3/users/swift` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:31.598447 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762dc81e-ff5d-49f8-b7de-6876a5adb8fe None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:31.600749 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762dc81e-ff5d-49f8-b7de-6876a5adb8fe None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:31.613181 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-762dc81e-ff5d-49f8-b7de-6876a5adb8fe None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 03 12:31:31.613181 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:31.613181 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:31.613181 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:31.613181 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:31.613181 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:31.613181 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:31.613181 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:31.613181 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:31.613181 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:31.613181 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:31.613181 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:31.613181 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 12:31:31.613181 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 12:31:31.613181 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 12:31:31.613181 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 12:31:31.614097 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:31.614097 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:31.614097 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 12:31:31.614097 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:31.614097 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 12:31:31.614097 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:31.614097 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:31.614097 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:31.614097 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:31.614097 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:31.614097 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:31.614097 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:31.614097 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:31.614097 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:31.614097 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:31.614097 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:31.614097 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:31.614097 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:31.615242 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:31.615242 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:31.615242 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 12:31:31.615242 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 12:31:31.615242 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 12:31:31.615242 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 12:31:31.615242 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 12:31:31.615242 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 12:31:31.615242 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 03 12:31:31.615242 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:31.615242 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 389/787] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 12:31:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:31.621608 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170c1751-22a4-4d0a-89fe-3ba7450d2dbc None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:31.740205 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170c1751-22a4-4d0a-89fe-3ba7450d2dbc None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:31.759357 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:31.792029 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e76d7-bf03-4b6e-99b8-db56e2886ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:31.792903 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e76d7-bf03-4b6e-99b8-db56e2886ad4 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:31.793348 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e76d7-bf03-4b6e-99b8-db56e2886ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:31.793850 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e76d7-bf03-4b6e-99b8-db56e2886ad4 None admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:31.794760 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e76d7-bf03-4b6e-99b8-db56e2886ad4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:31.795230 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e76d7-bf03-4b6e-99b8-db56e2886ad4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:31.796814 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e76d7-bf03-4b6e-99b8-db56e2886ad4 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:31.825591 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 399/788] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Oct 3 12:31:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:31:31.831188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762dc81e-ff5d-49f8-b7de-6876a5adb8fe None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:31.868791 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92015c63-6f8a-404e-b927-be2eb047f5b4 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:31.868982 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92015c63-6f8a-404e-b927-be2eb047f5b4 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:31.869252 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92015c63-6f8a-404e-b927-be2eb047f5b4 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:31.870836 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 400/789] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 12:31:31.878062 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1db34-b29f-44b5-af62-97ff977dc27e None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:31.878711 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1db34-b29f-44b5-af62-97ff977dc27e None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:31.879100 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1db34-b29f-44b5-af62-97ff977dc27e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:31.900770 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-37e1db34-b29f-44b5-af62-97ff977dc27e None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:31.902103 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-37e1db34-b29f-44b5-af62-97ff977dc27e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:32.004790 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762dc81e-ff5d-49f8-b7de-6876a5adb8fe None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:32.053592 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 401/790] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 12:31:32.072037 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d867bed1-653d-4b8f-9abf-f5f760c684b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:32.072690 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d867bed1-653d-4b8f-9abf-f5f760c684b1 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:32.072970 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d867bed1-653d-4b8f-9abf-f5f760c684b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:32.073242 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d867bed1-653d-4b8f-9abf-f5f760c684b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:32.104547 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d867bed1-653d-4b8f-9abf-f5f760c684b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:32.106007 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d867bed1-653d-4b8f-9abf-f5f760c684b1 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:32.129743 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-d867bed1-653d-4b8f-9abf-f5f760c684b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:32.129743 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:32.129743 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:32.129743 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:32.129743 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:32.129743 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:32.129743 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:32.129743 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:32.129743 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:32.129743 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:32.129743 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:32.129743 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:32.129743 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:32.129743 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:32.129743 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:32.129743 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:32.130921 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:32.130921 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:32.130921 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:32.130921 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:32.130921 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:32.130921 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:32.130921 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:32.130921 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:32.130921 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:32.130921 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:32.130921 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:32.130921 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:32.130921 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:32.130921 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:32.130921 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:32.130921 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:32.130921 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:32.132208 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:32.132208 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:32.132208 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:32.132208 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 390/791] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:32.136624 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e1db34-b29f-44b5-af62-97ff977dc27e None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:32.306098 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:32.309716 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e1db34-b29f-44b5-af62-97ff977dc27e None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:32.353631 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf8031-a919-400b-940c-e4f991475088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:32.354252 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf8031-a919-400b-940c-e4f991475088 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:32.354533 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf8031-a919-400b-940c-e4f991475088 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:32.354806 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf8031-a919-400b-940c-e4f991475088 None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:32.355476 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaf8031-a919-400b-940c-e4f991475088 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:32.355783 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaf8031-a919-400b-940c-e4f991475088 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:32.357147 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaf8031-a919-400b-940c-e4f991475088 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:32.372426 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 402/792] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:32.377963 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d867bed1-653d-4b8f-9abf-f5f760c684b1 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:32.381800 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:32.448198 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf8031-a919-400b-940c-e4f991475088 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:32.494544 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d867bed1-653d-4b8f-9abf-f5f760c684b1 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:32.533857 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e3f0c-5e9a-4456-892f-f9b62d490309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:32.534506 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e3f0c-5e9a-4456-892f-f9b62d490309 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:32.534832 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e3f0c-5e9a-4456-892f-f9b62d490309 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:32.535109 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e3f0c-5e9a-4456-892f-f9b62d490309 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:32.542791 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075e3f0c-5e9a-4456-892f-f9b62d490309 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:32.544287 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075e3f0c-5e9a-4456-892f-f9b62d490309 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:32.557368 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-075e3f0c-5e9a-4456-892f-f9b62d490309 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:32.557368 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:32.557368 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:32.557368 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:32.557368 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:32.557368 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:32.557368 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:32.557368 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:32.557368 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:32.557368 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:32.557368 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:32.557368 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:32.557368 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 12:31:32.557368 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 12:31:32.557368 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 12:31:32.558257 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 12:31:32.558257 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:32.558257 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:32.558257 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:32.558257 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:32.558257 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:32.558257 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:32.558257 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:32.558257 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:32.558257 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:32.558257 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:32.558257 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:32.558257 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:32.558257 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:32.558257 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:32.558257 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:32.558257 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:32.559148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 12:31:32.559148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 12:31:32.559148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 12:31:32.559148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 12:31:32.559148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 12:31:32.559148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 12:31:32.559148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:32.559148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:32.559148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 391/793] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:31:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:32.564301 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e3f0c-5e9a-4456-892f-f9b62d490309 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:32.637151 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:32.643788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf8031-a919-400b-940c-e4f991475088 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:32.679123 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82d9f46-4b7e-44ab-8687-8918b04d085b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:32.679533 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e3f0c-5e9a-4456-892f-f9b62d490309 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:32.679805 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d9f46-4b7e-44ab-8687-8918b04d085b None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:32.680087 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d9f46-4b7e-44ab-8687-8918b04d085b None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:32.680361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d9f46-4b7e-44ab-8687-8918b04d085b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:32.680945 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82d9f46-4b7e-44ab-8687-8918b04d085b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:32.682300 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82d9f46-4b7e-44ab-8687-8918b04d085b None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:32.693042 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-a82d9f46-4b7e-44ab-8687-8918b04d085b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 03 12:31:32.693042 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:32.693042 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 12:31:32.693042 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 12:31:32.693042 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 12:31:32.693042 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 12:31:32.693042 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 12:31:32.693042 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:31:32.693042 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 03 12:31:32.693042 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:32.693042 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 12:31:32.693042 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:32.693042 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:32.693042 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:32.693042 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:32.693042 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:32.693042 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:32.694311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:32.694311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:32.694311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:32.694311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:32.694311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:32.694311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:32.694311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 12:31:32.694311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 12:31:32.694311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 12:31:32.694311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 12:31:32.694311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:32.694311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:32.694311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:32.694311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:32.694311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:32.694311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:32.695149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:32.695149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:32.695149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:32.695149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:32.695149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:32.695149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:32.695149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:32.695149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:32.695149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:32.695149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:32.695149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 12:31:32.695149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:31:32.695149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 03 12:31:32.695149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:32.695149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 403/794] 10.222.0.33 () {56 vars in 1129 bytes} [Mon Oct 3 12:31:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:31:32.696008 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d53b4-0a84-4a41-a579-66fa8016d8cc None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:32.696395 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d53b4-0a84-4a41-a579-66fa8016d8cc None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:32.696496 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d53b4-0a84-4a41-a579-66fa8016d8cc None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:32.702238 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 404/795] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:32.703259 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653d53b4-0a84-4a41-a579-66fa8016d8cc None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:32.729037 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee0943-a46c-4d0d-9c02-940e5493666e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:32.729758 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee0943-a46c-4d0d-9c02-940e5493666e None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:32.730044 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee0943-a46c-4d0d-9c02-940e5493666e None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:32.730320 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee0943-a46c-4d0d-9c02-940e5493666e None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:32.731010 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee0943-a46c-4d0d-9c02-940e5493666e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:32.731336 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee0943-a46c-4d0d-9c02-940e5493666e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:32.732942 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:32.734039 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee0943-a46c-4d0d-9c02-940e5493666e None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:32.746966 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 392/796] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 12:31:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:32.749096 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580a03f-6db9-4c82-93a4-173a78789be7 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:32.749096 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580a03f-6db9-4c82-93a4-173a78789be7 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:32.749411 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580a03f-6db9-4c82-93a4-173a78789be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:32.772794 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-9580a03f-6db9-4c82-93a4-173a78789be7 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:32.774758 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-9580a03f-6db9-4c82-93a4-173a78789be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:32.819613 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653d53b4-0a84-4a41-a579-66fa8016d8cc None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:32.860338 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 393/797] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:31:32.861519 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9580a03f-6db9-4c82-93a4-173a78789be7 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:32.876040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349289f-c236-46e6-81c7-6a3244e40bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:32.876667 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349289f-c236-46e6-81c7-6a3244e40bfa None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:32.876947 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349289f-c236-46e6-81c7-6a3244e40bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:32.877220 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349289f-c236-46e6-81c7-6a3244e40bfa None admin] PATH_INFO: `/v3/services` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:32.877854 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0349289f-c236-46e6-81c7-6a3244e40bfa None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:32.878154 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0349289f-c236-46e6-81c7-6a3244e40bfa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:32.891719 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0349289f-c236-46e6-81c7-6a3244e40bfa None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:32.909326 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 405/798] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Oct 3 12:31:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:32.920280 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349289f-c236-46e6-81c7-6a3244e40bfa None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:32.925003 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:33.004798 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9580a03f-6db9-4c82-93a4-173a78789be7 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:33.048930 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333dd2d6-3143-4f17-9370-977dfa912de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:33.049601 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333dd2d6-3143-4f17-9370-977dfa912de9 None admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:33.049881 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333dd2d6-3143-4f17-9370-977dfa912de9 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:33.050154 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333dd2d6-3143-4f17-9370-977dfa912de9 None admin] PATH_INFO: `/v3/projects/4ea2fa8ad01240f7859182ebbe3051c7/users/b111d72a34144073917de4f566a58233/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:33.055252 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333dd2d6-3143-4f17-9370-977dfa912de9 None admin] RBAC: Authorizing `identity:create_grant(project_id=4ea2fa8ad01240f7859182ebbe3051c7, user_id=b111d72a34144073917de4f566a58233, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:33.063093 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333dd2d6-3143-4f17-9370-977dfa912de9 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:33.123352 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:33.136295 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349289f-c236-46e6-81c7-6a3244e40bfa None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:33.153841 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 394/799] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 12:31:32 2022] PUT /identity/v3/projects/4ea2fa8ad01240f7859182ebbe3051c7/users/b111d72a34144073917de4f566a58233/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:31:33.154777 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333dd2d6-3143-4f17-9370-977dfa912de9 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:33.194874 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4902de36-14b2-483f-b362-20201de3b550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:33.195972 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902de36-14b2-483f-b362-20201de3b550 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:33.197295 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902de36-14b2-483f-b362-20201de3b550 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:33.197871 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902de36-14b2-483f-b362-20201de3b550 None admin] PATH_INFO: `/v3/services` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:33.199103 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4902de36-14b2-483f-b362-20201de3b550 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:33.199778 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4902de36-14b2-483f-b362-20201de3b550 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:33.201432 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4902de36-14b2-483f-b362-20201de3b550 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:33.208171 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 406/800] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Oct 3 12:31:32 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:31:33.214845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4902de36-14b2-483f-b362-20201de3b550 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:33.385826 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333dd2d6-3143-4f17-9370-977dfa912de9 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:33.400735 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4902de36-14b2-483f-b362-20201de3b550 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:33.417361 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:33.445080 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed3d20-e4a5-4031-8e48-634ce6337d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:33.445705 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed3d20-e4a5-4031-8e48-634ce6337d81 None admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:33.445983 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed3d20-e4a5-4031-8e48-634ce6337d81 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:33.446253 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed3d20-e4a5-4031-8e48-634ce6337d81 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:33.446923 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ed3d20-e4a5-4031-8e48-634ce6337d81 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:33.448509 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ed3d20-e4a5-4031-8e48-634ce6337d81 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:33.452406 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b7e83-5f7b-4516-8e82-195be94e8fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:33.453598 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b7e83-5f7b-4516-8e82-195be94e8fc7 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:33.454091 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b7e83-5f7b-4516-8e82-195be94e8fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:33.455038 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b7e83-5f7b-4516-8e82-195be94e8fc7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:33.469382 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2b7e83-5f7b-4516-8e82-195be94e8fc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:31:33.474304 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-ef2b7e83-5f7b-4516-8e82-195be94e8fc7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:31:33.474304 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:33.474304 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:33.474304 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:33.474304 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:33.474304 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:33.474304 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:33.474304 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:33.474304 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:33.474304 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:33.474304 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:33.474304 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:33.474304 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:31:33.474304 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:31:33.474304 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:31:33.474304 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:31:33.476012 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:31:33.476012 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:31:33.476012 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:31:33.476012 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:31:33.476012 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:33.476012 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:33.476012 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:33.476012 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:33.476012 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:33.476012 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:33.476012 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:33.476012 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:33.476012 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:33.476012 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:33.476012 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:33.476012 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:33.476890 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:33.476890 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:33.476890 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:33.476890 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:33.476890 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:31:33.476890 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:31:33.476890 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:31:33.476890 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:31:33.476890 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:31:33.476890 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:31:33.476890 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:31:33.476890 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:33.476890 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 395/801] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 12:31:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:33.478052 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37749e8-03ca-49a3-9b60-19bca284ec81 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:33.478052 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37749e8-03ca-49a3-9b60-19bca284ec81 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:33.478209 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37749e8-03ca-49a3-9b60-19bca284ec81 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:33.487213 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 396/802] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:33.487213 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37749e8-03ca-49a3-9b60-19bca284ec81 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:33.537461 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 407/803] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:31:33 2022] POST /identity/v3/endpoints => generated 346 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:31:33.539635 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76940efd-b450-41be-b0f9-29e8c871a1ee None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:33.540102 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76940efd-b450-41be-b0f9-29e8c871a1ee None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:33.540525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76940efd-b450-41be-b0f9-29e8c871a1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:33.564725 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-76940efd-b450-41be-b0f9-29e8c871a1ee None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:33.566222 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-76940efd-b450-41be-b0f9-29e8c871a1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:33.569323 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:33.620144 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37749e8-03ca-49a3-9b60-19bca284ec81 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:33.700929 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6105afd-d59e-40e9-86cf-81a76457b89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:33.701817 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6105afd-d59e-40e9-86cf-81a76457b89f None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:33.702290 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6105afd-d59e-40e9-86cf-81a76457b89f None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:33.702785 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6105afd-d59e-40e9-86cf-81a76457b89f None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:33.703649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6105afd-d59e-40e9-86cf-81a76457b89f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:33.704117 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6105afd-d59e-40e9-86cf-81a76457b89f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:33.705733 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6105afd-d59e-40e9-86cf-81a76457b89f None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:33.734565 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 397/804] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:31:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:33.737629 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 408/805] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:31:33.738665 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6105afd-d59e-40e9-86cf-81a76457b89f None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:33.778027 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:33.855049 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6105afd-d59e-40e9-86cf-81a76457b89f None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:33.891630 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1413b84e-bfba-40fa-a078-7d3e1c9fb489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:33.893088 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413b84e-bfba-40fa-a078-7d3e1c9fb489 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:33.893570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413b84e-bfba-40fa-a078-7d3e1c9fb489 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:33.894046 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413b84e-bfba-40fa-a078-7d3e1c9fb489 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:33.905646 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1413b84e-bfba-40fa-a078-7d3e1c9fb489 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:33.907419 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1413b84e-bfba-40fa-a078-7d3e1c9fb489 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:33.924163 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-1413b84e-bfba-40fa-a078-7d3e1c9fb489 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:33.924163 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:33.924163 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:33.924163 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:33.924163 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:33.924163 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:33.924163 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:33.924163 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:33.924163 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:33.924163 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:33.924163 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:33.924163 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:33.924163 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:33.924163 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:33.924163 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:33.924163 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:33.925165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:33.925165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:33.925165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:33.925165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:33.925165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:33.925165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:33.925165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:33.925165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:33.925165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:33.925165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:33.925165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:33.925165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:33.925165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:33.925165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:33.925165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:33.925165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:33.925165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:33.926064 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:33.926064 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:33.926064 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:33.933476 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 398/806] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:33.936375 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76940efd-b450-41be-b0f9-29e8c871a1ee None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:34.049652 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76940efd-b450-41be-b0f9-29e8c871a1ee None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:34.072499 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1413b84e-bfba-40fa-a078-7d3e1c9fb489 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:34.086490 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec92bdb0-e105-4dbc-ae0c-ef9bfd58b816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:34.087107 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92bdb0-e105-4dbc-ae0c-ef9bfd58b816 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:34.087419 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92bdb0-e105-4dbc-ae0c-ef9bfd58b816 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:34.087692 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92bdb0-e105-4dbc-ae0c-ef9bfd58b816 None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:34.088330 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec92bdb0-e105-4dbc-ae0c-ef9bfd58b816 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:34.088631 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec92bdb0-e105-4dbc-ae0c-ef9bfd58b816 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:34.089989 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec92bdb0-e105-4dbc-ae0c-ef9bfd58b816 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:34.097200 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:34.103487 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 409/807] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:34.109279 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec92bdb0-e105-4dbc-ae0c-ef9bfd58b816 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:34.204632 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:34.224128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1413b84e-bfba-40fa-a078-7d3e1c9fb489 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:34.265729 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f4e9fb-d16b-4c68-bd7a-ed7394d656e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:34.266368 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4e9fb-d16b-4c68-bd7a-ed7394d656e9 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:34.266646 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4e9fb-d16b-4c68-bd7a-ed7394d656e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:34.266919 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4e9fb-d16b-4c68-bd7a-ed7394d656e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:34.267989 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec92bdb0-e105-4dbc-ae0c-ef9bfd58b816 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:34.278398 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f4e9fb-d16b-4c68-bd7a-ed7394d656e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:31:34.283169 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-28f4e9fb-d16b-4c68-bd7a-ed7394d656e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:31:34.283169 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:34.283169 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:34.283169 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:34.283169 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:34.283169 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:34.283169 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:34.283169 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:34.283169 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:34.283169 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:34.283169 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:34.283169 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:34.283169 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:31:34.283169 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:31:34.283169 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:31:34.283169 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:31:34.284117 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:31:34.284117 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:31:34.284117 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:31:34.284117 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:31:34.284117 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:34.284117 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:34.284117 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:34.284117 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:34.284117 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:34.284117 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:34.284117 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:34.284117 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:34.284117 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:34.284117 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:34.284117 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:34.284117 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:34.284936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:34.284936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:34.284936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:34.284936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:34.284936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:31:34.284936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:31:34.284936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:31:34.284936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:31:34.284936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:31:34.284936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:31:34.284936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:31:34.284936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:34.284936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 399/808] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 12:31:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:31:34.291874 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3acdafc-48c0-4746-bb9f-777d0bff4a37 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:34.291874 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3acdafc-48c0-4746-bb9f-777d0bff4a37 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:34.291874 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3acdafc-48c0-4746-bb9f-777d0bff4a37 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:34.291874 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 400/809] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:34.291874 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3acdafc-48c0-4746-bb9f-777d0bff4a37 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:34.328711 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe33506e-7f7b-46e3-ab8b-61e9c3eccefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:34.330444 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33506e-7f7b-46e3-ab8b-61e9c3eccefe None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:34.330935 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33506e-7f7b-46e3-ab8b-61e9c3eccefe None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:34.331464 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33506e-7f7b-46e3-ab8b-61e9c3eccefe None admin] PATH_INFO: `/v3/users/nova` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:34.342438 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe33506e-7f7b-46e3-ab8b-61e9c3eccefe None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:34.345279 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe33506e-7f7b-46e3-ab8b-61e9c3eccefe None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:34.361320 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-fe33506e-7f7b-46e3-ab8b-61e9c3eccefe None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 03 12:31:34.361320 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:34.361320 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:34.361320 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:34.361320 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:34.361320 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:34.361320 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:34.361320 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:34.361320 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:34.361320 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:34.361320 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:34.361320 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:34.361320 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 12:31:34.361320 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 12:31:34.361320 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 12:31:34.361320 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 12:31:34.362297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:34.362297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:34.362297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 12:31:34.362297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:34.362297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 12:31:34.362297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:34.362297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:34.362297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:34.362297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:34.362297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:34.362297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:34.362297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:34.362297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:34.362297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:34.362297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:34.362297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:34.362297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:34.362297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:34.363242 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:34.363242 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:34.363242 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 12:31:34.363242 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 12:31:34.363242 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 12:31:34.363242 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 12:31:34.363242 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 12:31:34.363242 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 12:31:34.363242 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 03 12:31:34.363242 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:34.363242 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 410/810] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Oct 3 12:31:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:31:34.363933 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265494f-9187-4a17-8501-7849bf63a79b None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:34.364623 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265494f-9187-4a17-8501-7849bf63a79b None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:34.364623 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265494f-9187-4a17-8501-7849bf63a79b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:34.391917 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-6265494f-9187-4a17-8501-7849bf63a79b None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:34.394197 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-6265494f-9187-4a17-8501-7849bf63a79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:34.412818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3acdafc-48c0-4746-bb9f-777d0bff4a37 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:34.451213 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea4a23b-bdb5-471e-a68e-ff695233be1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:34.451854 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4a23b-bdb5-471e-a68e-ff695233be1c None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:34.452132 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4a23b-bdb5-471e-a68e-ff695233be1c None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:34.452404 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4a23b-bdb5-471e-a68e-ff695233be1c None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:34.453062 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea4a23b-bdb5-471e-a68e-ff695233be1c None admin] RBAC: Adding query filter params (name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:34.453364 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea4a23b-bdb5-471e-a68e-ff695233be1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:34.454768 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea4a23b-bdb5-471e-a68e-ff695233be1c None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:34.468641 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 401/811] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 12:31:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:34.468641 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea4a23b-bdb5-471e-a68e-ff695233be1c None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:34.471015 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:34.479998 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:34.499399 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 411/812] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:31:34.500540 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6265494f-9187-4a17-8501-7849bf63a79b None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:34.633223 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea4a23b-bdb5-471e-a68e-ff695233be1c None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:34.676353 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56145f4-f8ac-4388-be3e-3c50893ad60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:34.677235 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56145f4-f8ac-4388-be3e-3c50893ad60a None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:34.677645 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56145f4-f8ac-4388-be3e-3c50893ad60a None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:34.677999 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56145f4-f8ac-4388-be3e-3c50893ad60a None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:34.678740 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56145f4-f8ac-4388-be3e-3c50893ad60a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:34.681648 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56145f4-f8ac-4388-be3e-3c50893ad60a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:34.681648 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56145f4-f8ac-4388-be3e-3c50893ad60a None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:34.690371 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6265494f-9187-4a17-8501-7849bf63a79b None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:34.705510 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 402/813] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Oct 3 12:31:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:34.707217 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56145f4-f8ac-4388-be3e-3c50893ad60a None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:34.763708 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b541a-2fa5-4b27-a67b-ae9377140f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:34.765588 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b541a-2fa5-4b27-a67b-ae9377140f4e None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:34.765871 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b541a-2fa5-4b27-a67b-ae9377140f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:34.766138 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b541a-2fa5-4b27-a67b-ae9377140f4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:34.786055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52b541a-2fa5-4b27-a67b-ae9377140f4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:34.788080 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52b541a-2fa5-4b27-a67b-ae9377140f4e None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:34.806106 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:34.816843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-c52b541a-2fa5-4b27-a67b-ae9377140f4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:34.816843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:34.816843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:34.816843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:34.816843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:34.816843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:34.816843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:34.816843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:34.816843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:34.816843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:34.816843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:34.816843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:34.816843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:34.816843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:34.816843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:34.816843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:34.820416 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:34.820416 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:34.820416 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:34.820416 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:34.820416 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:34.820416 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:34.820416 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:34.820416 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:34.820416 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:34.820416 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:34.820416 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:34.820416 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:34.820416 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:34.820416 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:34.820416 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:34.820416 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:34.820416 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:34.821293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:34.821293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:34.821293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:34.821293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 412/814] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:34.821293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b541a-2fa5-4b27-a67b-ae9377140f4e None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:34.861436 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56145f4-f8ac-4388-be3e-3c50893ad60a None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:34.870992 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:34.905360 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fee270-447e-4ed5-928c-d142f32ec3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:34.905913 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fee270-447e-4ed5-928c-d142f32ec3d7 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:34.906194 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fee270-447e-4ed5-928c-d142f32ec3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:34.906413 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fee270-447e-4ed5-928c-d142f32ec3d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:34.936035 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b541a-2fa5-4b27-a67b-ae9377140f4e None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:34.973671 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58435583-2fed-4bff-8872-3a56e63e32fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:34.975371 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58435583-2fed-4bff-8872-3a56e63e32fc None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:34.975864 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58435583-2fed-4bff-8872-3a56e63e32fc None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:34.976458 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58435583-2fed-4bff-8872-3a56e63e32fc None admin] PATH_INFO: `/v3/services/network` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:34.977340 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58435583-2fed-4bff-8872-3a56e63e32fc None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:34.979017 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58435583-2fed-4bff-8872-3a56e63e32fc None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:34.989049 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-58435583-2fed-4bff-8872-3a56e63e32fc None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 03 12:31:34.989049 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:34.989049 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 12:31:34.989049 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 12:31:34.989049 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 12:31:34.989049 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 12:31:34.989049 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 12:31:34.989049 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:31:34.989049 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 03 12:31:34.989049 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:34.989049 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 12:31:34.989049 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:34.989049 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:34.989049 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:34.989049 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:34.989049 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:34.989049 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:34.990059 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:34.990059 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:34.990059 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:34.990059 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:34.990059 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:34.990059 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:34.990059 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 12:31:34.990059 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 12:31:34.990059 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 12:31:34.990059 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 12:31:34.990059 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:34.990059 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:34.990059 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:34.990059 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:34.990059 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:34.990059 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:34.990888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:34.990888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:34.990888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:34.990888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:34.990888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:34.990888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:34.990888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:34.990888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:34.990888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:34.990888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:34.990888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 12:31:34.990888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:31:34.990888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 03 12:31:34.990888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:34.993190 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fee270-447e-4ed5-928c-d142f32ec3d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:34.994700 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 413/815] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:31:34 2022] GET /identity/v3/services/network => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:34.994977 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fee270-447e-4ed5-928c-d142f32ec3d7 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:34.995821 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58435583-2fed-4bff-8872-3a56e63e32fc None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:35.070289 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:35.117064 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58435583-2fed-4bff-8872-3a56e63e32fc None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:35.119578 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 403/816] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:31:34 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 12:31:35.120668 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:35.120809 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fee270-447e-4ed5-928c-d142f32ec3d7 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:35.168641 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caaf035-305c-4b96-a459-db67dbfc7589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:35.168641 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caaf035-305c-4b96-a459-db67dbfc7589 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:35.168641 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caaf035-305c-4b96-a459-db67dbfc7589 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:35.168641 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caaf035-305c-4b96-a459-db67dbfc7589 None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:35.169074 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caaf035-305c-4b96-a459-db67dbfc7589 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:35.169074 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caaf035-305c-4b96-a459-db67dbfc7589 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:35.169074 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caaf035-305c-4b96-a459-db67dbfc7589 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:35.182833 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 414/817] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:35.183706 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caaf035-305c-4b96-a459-db67dbfc7589 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:35.266318 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fee270-447e-4ed5-928c-d142f32ec3d7 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:35.311120 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35808d5-a005-4aca-80cb-180db3489e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:35.312000 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35808d5-a005-4aca-80cb-180db3489e41 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:35.312453 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35808d5-a005-4aca-80cb-180db3489e41 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:35.312870 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35808d5-a005-4aca-80cb-180db3489e41 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:35.313648 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35808d5-a005-4aca-80cb-180db3489e41 None admin] RBAC: Adding query filter params (name=network) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:35.314155 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35808d5-a005-4aca-80cb-180db3489e41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:35.314268 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caaf035-305c-4b96-a459-db67dbfc7589 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:35.315792 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35808d5-a005-4aca-80cb-180db3489e41 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:35.321402 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 404/818] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Oct 3 12:31:35 2022] GET /identity/v3/services?name=network => generated 124 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:35.322462 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35808d5-a005-4aca-80cb-180db3489e41 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:35.351719 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:35.355940 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b852f4e-7d51-4653-94ed-03928d8f2ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:35.356773 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b852f4e-7d51-4653-94ed-03928d8f2ae7 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:35.357185 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b852f4e-7d51-4653-94ed-03928d8f2ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:35.357576 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b852f4e-7d51-4653-94ed-03928d8f2ae7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:35.371490 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b852f4e-7d51-4653-94ed-03928d8f2ae7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:35.373038 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b852f4e-7d51-4653-94ed-03928d8f2ae7 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:35.390293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-8b852f4e-7d51-4653-94ed-03928d8f2ae7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:35.390293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:35.390293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:35.390293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:35.390293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:35.390293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:35.390293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:35.390293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:35.390293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:35.390293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:35.390293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:35.390293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:35.390293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:35.390293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:35.390293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:35.390293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:35.391491 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:35.391491 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:35.391491 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:35.391491 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:35.391491 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:35.391491 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:35.391491 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:35.391491 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:35.391491 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:35.391491 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:35.391491 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:35.391491 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:35.391491 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:35.391491 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:35.391491 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:35.391491 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:35.391491 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:35.392556 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:35.392556 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:35.392556 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:35.394045 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 415/819] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:35.395024 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b852f4e-7d51-4653-94ed-03928d8f2ae7 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:35.416133 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:35.432488 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35808d5-a005-4aca-80cb-180db3489e41 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:35.474181 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10327e4e-b7c2-4904-9b2b-ccdc62088c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:35.475792 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10327e4e-b7c2-4904-9b2b-ccdc62088c7a None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:35.476196 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10327e4e-b7c2-4904-9b2b-ccdc62088c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:35.476574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10327e4e-b7c2-4904-9b2b-ccdc62088c7a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:35.485829 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10327e4e-b7c2-4904-9b2b-ccdc62088c7a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:35.488337 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10327e4e-b7c2-4904-9b2b-ccdc62088c7a None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:35.506592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-10327e4e-b7c2-4904-9b2b-ccdc62088c7a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 12:31:35.506592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:35.506592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:35.506592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:35.506592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:35.506592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:35.506592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:35.506592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:35.506592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:35.506592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:35.506592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:35.506592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:35.506592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 12:31:35.506592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 12:31:35.506592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 12:31:35.506592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 12:31:35.508036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:35.508036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:35.508036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 12:31:35.508036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:35.508036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 12:31:35.508036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:35.508036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:35.508036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:35.508036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:35.508036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:35.508036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:35.508036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:35.508036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:35.508036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:35.508036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:35.508036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:35.508036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:35.508036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:35.509204 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:35.509204 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:35.509204 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 12:31:35.509204 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 12:31:35.509204 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 12:31:35.509204 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 12:31:35.509204 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 12:31:35.509204 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 12:31:35.509204 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 12:31:35.509204 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:35.511399 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 405/820] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Oct 3 12:31:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:35.512358 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10327e4e-b7c2-4904-9b2b-ccdc62088c7a None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:35.523310 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b852f4e-7d51-4653-94ed-03928d8f2ae7 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:35.565700 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2643a72-32b9-4aac-8006-0c232aca9705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:35.566728 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2643a72-32b9-4aac-8006-0c232aca9705 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:35.567402 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2643a72-32b9-4aac-8006-0c232aca9705 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:35.568440 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2643a72-32b9-4aac-8006-0c232aca9705 None admin] PATH_INFO: `/v3/services` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:35.569302 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2643a72-32b9-4aac-8006-0c232aca9705 None admin] RBAC: Adding query filter params (type=network) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:35.569993 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2643a72-32b9-4aac-8006-0c232aca9705 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:35.571631 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2643a72-32b9-4aac-8006-0c232aca9705 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:35.577667 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 416/821] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Oct 3 12:31:35 2022] GET /identity/v3/services?type=network => generated 350 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:35.579128 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2643a72-32b9-4aac-8006-0c232aca9705 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:35.619167 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:35.640382 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10327e4e-b7c2-4904-9b2b-ccdc62088c7a None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:35.677354 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:35.690253 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b59c208-c668-46ca-82f4-78c1bd71db9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:35.691699 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59c208-c668-46ca-82f4-78c1bd71db9e None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:35.692148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59c208-c668-46ca-82f4-78c1bd71db9e None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:35.692834 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59c208-c668-46ca-82f4-78c1bd71db9e None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:35.693745 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b59c208-c668-46ca-82f4-78c1bd71db9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:35.694368 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b59c208-c668-46ca-82f4-78c1bd71db9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:35.696692 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b59c208-c668-46ca-82f4-78c1bd71db9e None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:35.700934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2643a72-32b9-4aac-8006-0c232aca9705 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:35.714637 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 406/822] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:35.715708 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b59c208-c668-46ca-82f4-78c1bd71db9e None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:35.740591 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2baa2c-f211-42b6-857c-3987c7ceb709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:35.741607 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2baa2c-f211-42b6-857c-3987c7ceb709 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:35.741878 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2baa2c-f211-42b6-857c-3987c7ceb709 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:35.742304 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2baa2c-f211-42b6-857c-3987c7ceb709 None admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:35.743034 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2baa2c-f211-42b6-857c-3987c7ceb709 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:35.743382 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2baa2c-f211-42b6-857c-3987c7ceb709 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:35.745067 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2baa2c-f211-42b6-857c-3987c7ceb709 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:35.768782 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 417/823] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Oct 3 12:31:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:35.768782 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14822f04-2d73-4a8a-93b9-b00514797beb None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:35.768782 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14822f04-2d73-4a8a-93b9-b00514797beb None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:35.768782 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14822f04-2d73-4a8a-93b9-b00514797beb None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:35.769551 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 418/824] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:35.770325 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14822f04-2d73-4a8a-93b9-b00514797beb None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:35.828018 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b59c208-c668-46ca-82f4-78c1bd71db9e None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:35.862438 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58d937-d285-4136-b7c4-91c05c3c9613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:35.863049 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58d937-d285-4136-b7c4-91c05c3c9613 None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:35.863346 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58d937-d285-4136-b7c4-91c05c3c9613 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:35.864040 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58d937-d285-4136-b7c4-91c05c3c9613 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:35.864702 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e58d937-d285-4136-b7c4-91c05c3c9613 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:35.866624 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e58d937-d285-4136-b7c4-91c05c3c9613 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:35.876922 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14822f04-2d73-4a8a-93b9-b00514797beb None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:35.911211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdff6ad-f249-4fbc-ae47-9a7e3beb3412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:35.912446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdff6ad-f249-4fbc-ae47-9a7e3beb3412 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:35.912926 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdff6ad-f249-4fbc-ae47-9a7e3beb3412 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:35.913195 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdff6ad-f249-4fbc-ae47-9a7e3beb3412 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:35.919738 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdff6ad-f249-4fbc-ae47-9a7e3beb3412 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:35.921147 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdff6ad-f249-4fbc-ae47-9a7e3beb3412 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:35.922810 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:35.934286 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-0cdff6ad-f249-4fbc-ae47-9a7e3beb3412 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:35.934286 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:35.934286 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:35.934286 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:35.934286 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:35.934286 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:35.934286 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:35.934286 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:35.934286 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:35.934286 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:35.934286 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:35.934286 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:35.934286 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 12:31:35.934286 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 12:31:35.934286 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 12:31:35.937754 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 12:31:35.937754 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:35.937754 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:35.937754 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:35.937754 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:35.937754 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:35.937754 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:35.937754 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:35.937754 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:35.937754 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:35.937754 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:35.937754 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:35.937754 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:35.937754 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:35.937754 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:35.937754 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:35.937754 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:35.938660 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 12:31:35.938660 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 12:31:35.938660 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 12:31:35.938660 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 12:31:35.938660 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 12:31:35.938660 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 12:31:35.938660 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:35.938660 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:35.938660 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 419/825] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:31:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:35.938660 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdff6ad-f249-4fbc-ae47-9a7e3beb3412 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:35.972983 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 407/826] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:31:35 2022] POST /identity/v3/endpoints => generated 337 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:31:35.972983 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7939e8-7126-4c54-a60a-c121ab149c2c None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:35.972983 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7939e8-7126-4c54-a60a-c121ab149c2c None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:35.972983 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7939e8-7126-4c54-a60a-c121ab149c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:35.976944 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:35.995176 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-3a7939e8-7126-4c54-a60a-c121ab149c2c None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:35.997111 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-3a7939e8-7126-4c54-a60a-c121ab149c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:36.067122 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdff6ad-f249-4fbc-ae47-9a7e3beb3412 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:36.101869 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 408/827] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:35 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:31:36.102644 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7939e8-7126-4c54-a60a-c121ab149c2c None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:36.108000 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7328350b-f489-4557-bab3-35475d7b0e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:36.108637 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7328350b-f489-4557-bab3-35475d7b0e68 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:36.108914 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7328350b-f489-4557-bab3-35475d7b0e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:36.109184 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7328350b-f489-4557-bab3-35475d7b0e68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:36.125492 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7328350b-f489-4557-bab3-35475d7b0e68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:36.127247 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7328350b-f489-4557-bab3-35475d7b0e68 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:36.144980 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-7328350b-f489-4557-bab3-35475d7b0e68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:36.144980 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:36.144980 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:36.144980 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:36.144980 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:36.144980 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:36.144980 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:36.144980 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:36.144980 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:36.144980 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:36.144980 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:36.144980 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:36.144980 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:36.144980 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:36.144980 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:36.144980 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:36.146011 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:36.146011 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:36.146011 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:36.146011 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:36.146011 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:36.146011 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:36.146011 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:36.146011 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:36.146011 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:36.146011 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:36.146011 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:36.146011 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:36.146011 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:36.146011 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:36.146011 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:36.146011 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:36.146011 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:36.147200 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:36.147200 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:36.147200 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:36.147200 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 420/828] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:36.148896 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a1e28-0ab5-401f-8578-5b20f6a1c998 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:36.148896 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a1e28-0ab5-401f-8578-5b20f6a1c998 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:36.148896 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a1e28-0ab5-401f-8578-5b20f6a1c998 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:36.154038 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 421/829] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:36.154942 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94a1e28-0ab5-401f-8578-5b20f6a1c998 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:36.225858 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:36.246884 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7939e8-7126-4c54-a60a-c121ab149c2c None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:36.281619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:36.290520 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c34c74-f5d7-49d4-961b-243b5e487510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:36.291148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c34c74-f5d7-49d4-961b-243b5e487510 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:36.291463 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c34c74-f5d7-49d4-961b-243b5e487510 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:36.291739 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c34c74-f5d7-49d4-961b-243b5e487510 None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:36.292431 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c34c74-f5d7-49d4-961b-243b5e487510 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:36.292736 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c34c74-f5d7-49d4-961b-243b5e487510 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:36.294154 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c34c74-f5d7-49d4-961b-243b5e487510 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:36.300043 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94a1e28-0ab5-401f-8578-5b20f6a1c998 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:36.306941 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 409/830] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 12:31:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:36.309167 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e885c46-cd04-4d47-b3a4-134fc13e8866 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:36.309691 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e885c46-cd04-4d47-b3a4-134fc13e8866 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:36.310175 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e885c46-cd04-4d47-b3a4-134fc13e8866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:36.329726 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-6e885c46-cd04-4d47-b3a4-134fc13e8866 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:36.331073 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-6e885c46-cd04-4d47-b3a4-134fc13e8866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:36.337019 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b5596-34fe-449f-93c7-cae9b409905f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:36.337824 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b5596-34fe-449f-93c7-cae9b409905f None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:36.338294 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b5596-34fe-449f-93c7-cae9b409905f None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:36.338705 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b5596-34fe-449f-93c7-cae9b409905f None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:36.339507 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25b5596-34fe-449f-93c7-cae9b409905f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:36.339952 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25b5596-34fe-449f-93c7-cae9b409905f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:36.341459 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25b5596-34fe-449f-93c7-cae9b409905f None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:36.354391 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 422/831] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:36.355473 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b5596-34fe-449f-93c7-cae9b409905f None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:36.407131 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 410/832] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:36 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:31:36.408003 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e885c46-cd04-4d47-b3a4-134fc13e8866 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:36.461403 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b5596-34fe-449f-93c7-cae9b409905f None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:36.472550 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:36.497710 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8acf5a7-8bf4-4bc9-ad7a-02343bd12824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:36.498684 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8acf5a7-8bf4-4bc9-ad7a-02343bd12824 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:36.498975 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8acf5a7-8bf4-4bc9-ad7a-02343bd12824 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:36.499253 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8acf5a7-8bf4-4bc9-ad7a-02343bd12824 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:36.500025 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8acf5a7-8bf4-4bc9-ad7a-02343bd12824 None admin] RBAC: Adding query filter params (role.id=7933c45dcdd54ef0a98018ba8a8bddaa, scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7, user.id=990f5837994f4a088c656c9b33e9a0dc) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:36.500340 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8acf5a7-8bf4-4bc9-ad7a-02343bd12824 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:36.501768 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8acf5a7-8bf4-4bc9-ad7a-02343bd12824 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:36.508819 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 423/833] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:31:36 2022] GET /identity/v3/role_assignments?user.id=990f5837994f4a088c656c9b33e9a0dc&scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7&role.id=7933c45dcdd54ef0a98018ba8a8bddaa => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:36.509605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8acf5a7-8bf4-4bc9-ad7a-02343bd12824 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:36.529363 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e885c46-cd04-4d47-b3a4-134fc13e8866 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:36.550273 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:36.569593 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eacab8-b71d-4cae-a093-d71fc2f97768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:36.570741 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eacab8-b71d-4cae-a093-d71fc2f97768 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:36.571201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eacab8-b71d-4cae-a093-d71fc2f97768 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:36.571509 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eacab8-b71d-4cae-a093-d71fc2f97768 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:36.579387 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eacab8-b71d-4cae-a093-d71fc2f97768 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:36.580965 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eacab8-b71d-4cae-a093-d71fc2f97768 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:36.596806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-48eacab8-b71d-4cae-a093-d71fc2f97768 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:36.596806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:36.596806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:36.596806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:36.596806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:36.596806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:36.596806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:36.596806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:36.596806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:36.596806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:36.596806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:36.596806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:36.596806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 12:31:36.596806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 12:31:36.596806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 12:31:36.597784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 12:31:36.597784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:36.597784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:36.597784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:36.597784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:36.597784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:36.597784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:36.597784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:36.597784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:36.597784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:36.597784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:36.597784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:36.597784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:36.597784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:36.597784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:36.597784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:36.597784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:36.598682 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 12:31:36.598682 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 12:31:36.598682 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 12:31:36.598682 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 12:31:36.598682 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 12:31:36.598682 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 12:31:36.598682 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:36.598682 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:36.603363 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 411/834] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:31:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:36.605185 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eacab8-b71d-4cae-a093-d71fc2f97768 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:36.664992 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8acf5a7-8bf4-4bc9-ad7a-02343bd12824 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:36.711849 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e1cce4-2c9a-423f-91b8-207822c84b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:36.712763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1cce4-2c9a-423f-91b8-207822c84b69 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:36.714058 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1cce4-2c9a-423f-91b8-207822c84b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:36.714058 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1cce4-2c9a-423f-91b8-207822c84b69 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:36.723697 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eacab8-b71d-4cae-a093-d71fc2f97768 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:36.725971 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e1cce4-2c9a-423f-91b8-207822c84b69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:31:36.731300 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-a9e1cce4-2c9a-423f-91b8-207822c84b69 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:31:36.731300 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:36.731300 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:36.731300 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:36.731300 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:36.731300 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:36.731300 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:36.731300 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:36.731300 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:36.731300 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:36.731300 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:36.731300 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:36.731300 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:31:36.731300 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:31:36.731300 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:31:36.731300 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:31:36.732232 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:31:36.732232 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:31:36.732232 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:31:36.732232 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:31:36.732232 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:36.732232 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:36.732232 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:36.732232 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:36.732232 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:36.732232 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:36.732232 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:36.732232 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:36.732232 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:36.732232 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:36.732232 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:36.732232 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:36.733069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:36.733069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:36.733069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:36.733069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:36.733069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:31:36.733069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:31:36.733069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:31:36.733069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:31:36.733069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:31:36.733069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:31:36.733069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:31:36.733069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:36.733069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 424/835] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 12:31:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:36.733754 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e1cce4-2c9a-423f-91b8-207822c84b69 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:36.768581 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a5b959-abad-430f-8d6d-49095b30fbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:36.769130 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a5b959-abad-430f-8d6d-49095b30fbd2 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:36.769409 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a5b959-abad-430f-8d6d-49095b30fbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:36.769910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a5b959-abad-430f-8d6d-49095b30fbd2 None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:36.770364 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a5b959-abad-430f-8d6d-49095b30fbd2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:36.770667 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a5b959-abad-430f-8d6d-49095b30fbd2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:36.773214 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a5b959-abad-430f-8d6d-49095b30fbd2 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:36.776640 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:36.785733 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 412/836] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 12:31:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:31:36.793035 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a5b959-abad-430f-8d6d-49095b30fbd2 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:36.841416 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:36.844833 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e1cce4-2c9a-423f-91b8-207822c84b69 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:36.882144 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e486347-5fde-4ea1-80bc-9a10c05d7591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:36.882801 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e486347-5fde-4ea1-80bc-9a10c05d7591 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:36.883076 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e486347-5fde-4ea1-80bc-9a10c05d7591 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:36.883359 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e486347-5fde-4ea1-80bc-9a10c05d7591 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:36.883967 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e486347-5fde-4ea1-80bc-9a10c05d7591 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:36.885332 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e486347-5fde-4ea1-80bc-9a10c05d7591 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:36.895216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-9e486347-5fde-4ea1-80bc-9a10c05d7591 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 03 12:31:36.895216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:36.895216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 12:31:36.895216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 12:31:36.895216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 12:31:36.895216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 12:31:36.895216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 12:31:36.895216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:31:36.895216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 03 12:31:36.895216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:36.895216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 12:31:36.895216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:36.895216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:36.895216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:36.895216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:36.895216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:36.895216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:36.896803 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:36.896803 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:36.896803 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:36.896803 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:36.896803 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:36.896803 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:36.896803 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 12:31:36.896803 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 12:31:36.896803 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 12:31:36.896803 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 12:31:36.896803 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:36.896803 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:36.896803 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:36.896803 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:36.896803 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:36.896803 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:36.897666 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:36.897666 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:36.897666 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:36.897666 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:36.897666 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:36.897666 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:36.897666 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:36.897666 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:36.897666 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:36.897666 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:36.897666 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 12:31:36.897666 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:31:36.897666 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 03 12:31:36.897666 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:36.897666 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 425/837] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 12:31:36 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:36.898459 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e486347-5fde-4ea1-80bc-9a10c05d7591 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:36.914603 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a5b959-abad-430f-8d6d-49095b30fbd2 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:36.955033 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51566366-03f4-4ad7-a15e-0bb148e44dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:36.955689 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51566366-03f4-4ad7-a15e-0bb148e44dab None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:36.955971 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51566366-03f4-4ad7-a15e-0bb148e44dab None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:36.956244 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51566366-03f4-4ad7-a15e-0bb148e44dab None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:36.956907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51566366-03f4-4ad7-a15e-0bb148e44dab None admin] RBAC: Adding query filter params (name=admin) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:36.957208 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51566366-03f4-4ad7-a15e-0bb148e44dab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:36.958567 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51566366-03f4-4ad7-a15e-0bb148e44dab None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:36.968171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 413/838] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:31:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:36.968985 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51566366-03f4-4ad7-a15e-0bb148e44dab None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:37.009563 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:37.012931 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e486347-5fde-4ea1-80bc-9a10c05d7591 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:37.049042 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c641d-5a1c-47c4-88e0-c9a67605752d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:37.049619 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c641d-5a1c-47c4-88e0-c9a67605752d None admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:37.049895 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c641d-5a1c-47c4-88e0-c9a67605752d None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:37.050166 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c641d-5a1c-47c4-88e0-c9a67605752d None admin] PATH_INFO: `/v3/projects/4ea2fa8ad01240f7859182ebbe3051c7/users/2e6e4387a7c24bbc82c22eefe2a3d138/roles/ff6b2d4413644db39a2bd8576afc6ccb` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:37.072824 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c641d-5a1c-47c4-88e0-c9a67605752d None admin] RBAC: Authorizing `identity:create_grant(project_id=4ea2fa8ad01240f7859182ebbe3051c7, user_id=2e6e4387a7c24bbc82c22eefe2a3d138, role_id=ff6b2d4413644db39a2bd8576afc6ccb)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:37.074258 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c641d-5a1c-47c4-88e0-c9a67605752d None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:37.078694 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:37.081564 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51566366-03f4-4ad7-a15e-0bb148e44dab None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:37.118713 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d090ba-6794-4221-b169-0a5bced0fbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:37.119337 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d090ba-6794-4221-b169-0a5bced0fbb0 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:37.119655 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d090ba-6794-4221-b169-0a5bced0fbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:37.119929 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d090ba-6794-4221-b169-0a5bced0fbb0 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:37.120553 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d090ba-6794-4221-b169-0a5bced0fbb0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:37.120851 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d090ba-6794-4221-b169-0a5bced0fbb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:37.122211 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d090ba-6794-4221-b169-0a5bced0fbb0 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:37.126758 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 414/839] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 12:31:36 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:37.127566 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d090ba-6794-4221-b169-0a5bced0fbb0 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:37.167344 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 426/840] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 12:31:36 2022] PUT /identity/v3/projects/4ea2fa8ad01240f7859182ebbe3051c7/users/2e6e4387a7c24bbc82c22eefe2a3d138/roles/ff6b2d4413644db39a2bd8576afc6ccb => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:31:37.167344 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c641d-5a1c-47c4-88e0-c9a67605752d None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:37.272771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d090ba-6794-4221-b169-0a5bced0fbb0 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:37.295882 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c641d-5a1c-47c4-88e0-c9a67605752d None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:37.313555 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45c703-364a-415e-b08f-f098b4c556d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:37.314321 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45c703-364a-415e-b08f-f098b4c556d9 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:37.314669 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45c703-364a-415e-b08f-f098b4c556d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:37.315009 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45c703-364a-415e-b08f-f098b4c556d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:37.328427 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec45c703-364a-415e-b08f-f098b4c556d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:37.330083 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec45c703-364a-415e-b08f-f098b4c556d9 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:37.337241 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876360f-ceb7-4fad-8341-78aafc2536e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:37.337876 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876360f-ceb7-4fad-8341-78aafc2536e0 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:37.338739 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876360f-ceb7-4fad-8341-78aafc2536e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:37.339011 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876360f-ceb7-4fad-8341-78aafc2536e0 None admin] PATH_INFO: `/v3/services` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:37.339686 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6876360f-ceb7-4fad-8341-78aafc2536e0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:37.339991 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6876360f-ceb7-4fad-8341-78aafc2536e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:37.342160 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6876360f-ceb7-4fad-8341-78aafc2536e0 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:37.348479 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-ec45c703-364a-415e-b08f-f098b4c556d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:37.348479 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:37.348479 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:37.348479 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:37.348479 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:37.348479 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:37.348479 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:37.348479 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:37.348479 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:37.348479 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:37.348479 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:37.348479 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:37.348479 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:37.348479 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:37.348479 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:37.348479 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:37.350395 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:37.350395 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:37.350395 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:37.350395 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:37.350395 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:37.350395 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:37.350395 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:37.350395 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:37.350395 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:37.350395 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:37.350395 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:37.350395 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:37.350395 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:37.350395 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:37.350395 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:37.350395 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:37.350395 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:37.352114 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:37.352114 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:37.352114 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:37.352406 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 427/841] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 12:31:37 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:37.356656 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 415/842] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:37.361649 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876360f-ceb7-4fad-8341-78aafc2536e0 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:37.379182 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:37.480316 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876360f-ceb7-4fad-8341-78aafc2536e0 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:37.521345 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0f7b2-5b74-4e11-a835-534a8771ef26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:37.522178 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0f7b2-5b74-4e11-a835-534a8771ef26 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:37.522515 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0f7b2-5b74-4e11-a835-534a8771ef26 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:37.522796 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0f7b2-5b74-4e11-a835-534a8771ef26 None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:37.523479 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc0f7b2-5b74-4e11-a835-534a8771ef26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:37.523876 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc0f7b2-5b74-4e11-a835-534a8771ef26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:37.525240 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc0f7b2-5b74-4e11-a835-534a8771ef26 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:37.538382 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 428/843] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:37.543563 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45c703-364a-415e-b08f-f098b4c556d9 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:37.569772 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:37.665755 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45c703-364a-415e-b08f-f098b4c556d9 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:37.703688 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163c7ed-2370-4dc7-a51a-1a712cc2183b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:37.704704 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163c7ed-2370-4dc7-a51a-1a712cc2183b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:37.705204 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163c7ed-2370-4dc7-a51a-1a712cc2183b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:37.705677 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163c7ed-2370-4dc7-a51a-1a712cc2183b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:37.714981 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2163c7ed-2370-4dc7-a51a-1a712cc2183b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:37.716814 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2163c7ed-2370-4dc7-a51a-1a712cc2183b None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:37.730404 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-2163c7ed-2370-4dc7-a51a-1a712cc2183b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 03 12:31:37.730404 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:37.730404 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:37.730404 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:37.730404 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:37.730404 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:37.730404 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:37.730404 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:37.730404 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:37.730404 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:37.730404 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:37.730404 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:37.730404 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 12:31:37.730404 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 12:31:37.730404 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 12:31:37.730404 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 12:31:37.731451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:37.731451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:37.731451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 12:31:37.731451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:37.731451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 12:31:37.731451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:37.731451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:37.731451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:37.731451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:37.731451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:37.731451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:37.731451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:37.731451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:37.731451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:37.731451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:37.731451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:37.731451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:37.731451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:37.732411 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:37.732411 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:37.732411 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 12:31:37.732411 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 12:31:37.732411 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 12:31:37.732411 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 12:31:37.732411 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 12:31:37.732411 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 12:31:37.732411 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 03 12:31:37.732411 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:37.736151 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 416/844] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 12:31:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:31:37.744450 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0f7b2-5b74-4e11-a835-534a8771ef26 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:37.823051 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:37.855594 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0f7b2-5b74-4e11-a835-534a8771ef26 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:37.893484 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f58388-79d8-407a-a59c-16dbb297ece1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:37.894103 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f58388-79d8-407a-a59c-16dbb297ece1 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:37.894386 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f58388-79d8-407a-a59c-16dbb297ece1 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:37.894659 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f58388-79d8-407a-a59c-16dbb297ece1 None admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:37.895429 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f58388-79d8-407a-a59c-16dbb297ece1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:37.895745 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f58388-79d8-407a-a59c-16dbb297ece1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:37.898547 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f58388-79d8-407a-a59c-16dbb297ece1 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:37.917697 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 429/845] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Oct 3 12:31:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:37.923391 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163c7ed-2370-4dc7-a51a-1a712cc2183b None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:38.011195 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:38.035728 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163c7ed-2370-4dc7-a51a-1a712cc2183b None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:38.077529 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234289de-4930-4a39-ab16-694313e97c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:38.078494 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234289de-4930-4a39-ab16-694313e97c9e None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:38.078968 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234289de-4930-4a39-ab16-694313e97c9e None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:38.079906 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234289de-4930-4a39-ab16-694313e97c9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:38.151205 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234289de-4930-4a39-ab16-694313e97c9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:38.153082 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234289de-4930-4a39-ab16-694313e97c9e None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:38.247495 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:38.276981 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 417/846] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:31:37 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 12:31:38.282705 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f58388-79d8-407a-a59c-16dbb297ece1 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:38.397692 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f58388-79d8-407a-a59c-16dbb297ece1 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:38.430839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:38.438328 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c03ba90-17b1-4b05-9dd1-2600680f6767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:38.439198 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03ba90-17b1-4b05-9dd1-2600680f6767 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:38.439750 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03ba90-17b1-4b05-9dd1-2600680f6767 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:38.440333 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03ba90-17b1-4b05-9dd1-2600680f6767 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:38.451478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c03ba90-17b1-4b05-9dd1-2600680f6767 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:38.453215 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c03ba90-17b1-4b05-9dd1-2600680f6767 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:38.469105 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-3c03ba90-17b1-4b05-9dd1-2600680f6767 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:38.469105 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:38.469105 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:38.469105 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:38.469105 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:38.469105 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:38.469105 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:38.469105 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:38.469105 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:38.469105 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:38.469105 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:38.469105 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:38.469105 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:38.469105 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:38.469105 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:38.469105 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:38.470071 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:38.470071 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:38.470071 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:38.470071 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:38.470071 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:38.470071 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:38.470071 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:38.470071 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:38.470071 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:38.470071 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:38.470071 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:38.470071 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:38.470071 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:38.470071 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:38.470071 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:38.470071 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:38.470071 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:38.470958 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:38.470958 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:38.470958 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:38.472789 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 430/847] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:38.479412 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234289de-4930-4a39-ab16-694313e97c9e None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:38.584882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234289de-4930-4a39-ab16-694313e97c9e None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:38.622666 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564c82b0-408e-4849-9ebe-f115b81572e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:38.624103 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c82b0-408e-4849-9ebe-f115b81572e1 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:38.624786 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c82b0-408e-4849-9ebe-f115b81572e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:38.625212 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c82b0-408e-4849-9ebe-f115b81572e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:38.626006 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564c82b0-408e-4849-9ebe-f115b81572e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:38.626472 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564c82b0-408e-4849-9ebe-f115b81572e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:38.628013 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564c82b0-408e-4849-9ebe-f115b81572e1 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:38.641727 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 418/848] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:38.647826 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c03ba90-17b1-4b05-9dd1-2600680f6767 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:38.759360 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c03ba90-17b1-4b05-9dd1-2600680f6767 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:38.795746 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fab0cc-d04d-49fd-b0d0-db31c84a8cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:38.796377 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fab0cc-d04d-49fd-b0d0-db31c84a8cf8 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:38.796652 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fab0cc-d04d-49fd-b0d0-db31c84a8cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:38.796915 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fab0cc-d04d-49fd-b0d0-db31c84a8cf8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:38.804492 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fab0cc-d04d-49fd-b0d0-db31c84a8cf8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:38.805920 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fab0cc-d04d-49fd-b0d0-db31c84a8cf8 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:38.817346 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-c9fab0cc-d04d-49fd-b0d0-db31c84a8cf8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:38.817346 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:38.817346 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:38.817346 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:38.817346 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:38.817346 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:38.817346 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:38.817346 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:38.817346 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:38.817346 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:38.817346 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:38.817346 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:38.817346 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 12:31:38.817346 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 12:31:38.817346 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 12:31:38.819234 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 12:31:38.819234 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:38.819234 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:38.819234 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:38.819234 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:38.819234 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:38.819234 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:38.819234 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:38.819234 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:38.819234 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:38.819234 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:38.819234 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:38.819234 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:38.819234 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:38.819234 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:38.819234 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:38.819234 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:38.820230 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 12:31:38.820230 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 12:31:38.820230 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 12:31:38.820230 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 12:31:38.820230 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 12:31:38.820230 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 12:31:38.820230 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:38.820230 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:38.820230 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 431/849] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:31:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:31:38.828274 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564c82b0-408e-4849-9ebe-f115b81572e1 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:38.864018 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc2040-abe4-402e-88f1-d0cb6841a420 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:38.865598 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc2040-abe4-402e-88f1-d0cb6841a420 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:38.866131 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc2040-abe4-402e-88f1-d0cb6841a420 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:38.867990 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 432/850] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 12:31:38.873167 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522efa09-41c7-4b1a-82ff-42ac75a21fd9 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:38.873448 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522efa09-41c7-4b1a-82ff-42ac75a21fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:38.873722 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522efa09-41c7-4b1a-82ff-42ac75a21fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:38.875224 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:38.895303 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-522efa09-41c7-4b1a-82ff-42ac75a21fd9 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:38.897081 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-522efa09-41c7-4b1a-82ff-42ac75a21fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:38.939766 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:38.940431 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564c82b0-408e-4849-9ebe-f115b81572e1 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:38.979583 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 433/851] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:38 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 12:31:38.983768 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3cc458-3311-4327-bf1d-314424b15293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:38.984567 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3cc458-3311-4327-bf1d-314424b15293 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:38.984976 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3cc458-3311-4327-bf1d-314424b15293 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:38.985378 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3cc458-3311-4327-bf1d-314424b15293 None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:38.986202 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3cc458-3311-4327-bf1d-314424b15293 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:38.986646 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3cc458-3311-4327-bf1d-314424b15293 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:38.988275 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3cc458-3311-4327-bf1d-314424b15293 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:39.001164 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 419/852] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 12:31:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:39.008036 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522efa09-41c7-4b1a-82ff-42ac75a21fd9 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:39.122009 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522efa09-41c7-4b1a-82ff-42ac75a21fd9 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:39.164499 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c1d24-b805-4e82-b8ab-88ca943c537d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:39.165546 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c1d24-b805-4e82-b8ab-88ca943c537d None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:39.166092 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c1d24-b805-4e82-b8ab-88ca943c537d None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:39.166570 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c1d24-b805-4e82-b8ab-88ca943c537d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:39.167537 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c1d24-b805-4e82-b8ab-88ca943c537d None admin] RBAC: Adding query filter params (role.id=7933c45dcdd54ef0a98018ba8a8bddaa, scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7, user.id=b111d72a34144073917de4f566a58233) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:39.168075 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c1d24-b805-4e82-b8ab-88ca943c537d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:39.169710 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c1d24-b805-4e82-b8ab-88ca943c537d None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:39.176688 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 434/853] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:31:39 2022] GET /identity/v3/role_assignments?user.id=b111d72a34144073917de4f566a58233&scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7&role.id=7933c45dcdd54ef0a98018ba8a8bddaa => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:39.265755 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3cc458-3311-4327-bf1d-314424b15293 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:39.378472 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:39.385896 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3cc458-3311-4327-bf1d-314424b15293 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:39.421547 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b515fc-3121-4ea8-8d20-0a02159dbfb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:39.422566 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b515fc-3121-4ea8-8d20-0a02159dbfb3 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:39.423102 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b515fc-3121-4ea8-8d20-0a02159dbfb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:39.423707 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b515fc-3121-4ea8-8d20-0a02159dbfb3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:39.424537 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b515fc-3121-4ea8-8d20-0a02159dbfb3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:39.426095 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b515fc-3121-4ea8-8d20-0a02159dbfb3 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:39.436548 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-83b515fc-3121-4ea8-8d20-0a02159dbfb3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 03 12:31:39.436548 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:39.436548 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 12:31:39.436548 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 12:31:39.436548 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 12:31:39.436548 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 12:31:39.436548 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 12:31:39.436548 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:31:39.436548 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 03 12:31:39.436548 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:39.436548 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 12:31:39.436548 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:39.436548 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:39.436548 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:39.436548 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:39.436548 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:39.437532 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:39.437532 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:39.437532 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:39.437532 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:39.437532 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:39.437532 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:39.437532 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:39.437532 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 12:31:39.437532 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 12:31:39.437532 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 12:31:39.437532 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 12:31:39.437532 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:39.437532 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:39.437532 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:39.437532 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:39.437532 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:39.437532 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:39.438415 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:39.438415 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:39.438415 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:39.438415 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:39.438415 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:39.438415 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:39.438415 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:39.438415 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:39.438415 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:39.438415 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:39.438415 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 12:31:39.438415 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:31:39.438415 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 03 12:31:39.438415 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:39.441431 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 420/854] 10.222.0.33 () {56 vars in 1132 bytes} [Mon Oct 3 12:31:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:39.447698 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c1d24-b805-4e82-b8ab-88ca943c537d None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:39.491883 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:39.560358 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c1d24-b805-4e82-b8ab-88ca943c537d None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:39.575555 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ac87b-75b7-44e7-97bd-2e93da325153 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:39.575893 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ac87b-75b7-44e7-97bd-2e93da325153 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:39.576106 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ac87b-75b7-44e7-97bd-2e93da325153 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:39.577746 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 421/855] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 12:31:39.582699 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414ff6c-1897-4d2b-b4f3-9e87cd268b26 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:39.582978 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414ff6c-1897-4d2b-b4f3-9e87cd268b26 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:39.583381 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414ff6c-1897-4d2b-b4f3-9e87cd268b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:39.597474 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c34b167-39c5-4c6e-92bb-cb42995e3066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:39.598250 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34b167-39c5-4c6e-92bb-cb42995e3066 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:39.598537 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34b167-39c5-4c6e-92bb-cb42995e3066 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:39.598812 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34b167-39c5-4c6e-92bb-cb42995e3066 None admin] PATH_INFO: `/v3/services` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:39.599564 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c34b167-39c5-4c6e-92bb-cb42995e3066 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:39.599878 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c34b167-39c5-4c6e-92bb-cb42995e3066 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:39.601305 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c34b167-39c5-4c6e-92bb-cb42995e3066 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:39.605749 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-2414ff6c-1897-4d2b-b4f3-9e87cd268b26 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:39.606931 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 435/856] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Oct 3 12:31:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:39.607582 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-2414ff6c-1897-4d2b-b4f3-9e87cd268b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:39.613838 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c34b167-39c5-4c6e-92bb-cb42995e3066 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:39.690513 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 422/857] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:39 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 12:31:39.719536 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42a940-e202-4eda-a69f-3f8af33a32e8 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:39.719858 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42a940-e202-4eda-a69f-3f8af33a32e8 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:39.720086 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42a940-e202-4eda-a69f-3f8af33a32e8 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:39.721659 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 423/858] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:39.728615 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395db4e-b529-4d35-a321-771ebcdc4910 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:39.729152 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395db4e-b529-4d35-a321-771ebcdc4910 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:39.729152 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395db4e-b529-4d35-a321-771ebcdc4910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:39.732149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:39.738415 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c34b167-39c5-4c6e-92bb-cb42995e3066 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:39.748529 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-6395db4e-b529-4d35-a321-771ebcdc4910 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:39.749993 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-6395db4e-b529-4d35-a321-771ebcdc4910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:39.775419 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda13f4f-0525-48b7-b15d-4cb885386f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:39.776750 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda13f4f-0525-48b7-b15d-4cb885386f2a None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:39.777232 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda13f4f-0525-48b7-b15d-4cb885386f2a None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:39.777715 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda13f4f-0525-48b7-b15d-4cb885386f2a None admin] PATH_INFO: `/v3/services` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:39.778462 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda13f4f-0525-48b7-b15d-4cb885386f2a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:39.778838 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda13f4f-0525-48b7-b15d-4cb885386f2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:39.780300 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda13f4f-0525-48b7-b15d-4cb885386f2a None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:39.785322 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 436/859] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Oct 3 12:31:39 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:39.837352 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 424/860] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:39 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:31:39.978746 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda13f4f-0525-48b7-b15d-4cb885386f2a None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:40.090129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda13f4f-0525-48b7-b15d-4cb885386f2a None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:40.117043 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6395db4e-b529-4d35-a321-771ebcdc4910 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:40.140532 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:40.141583 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7bf64a-2b35-45d1-b049-ea3cc6d16800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:40.142508 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7bf64a-2b35-45d1-b049-ea3cc6d16800 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:40.142789 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7bf64a-2b35-45d1-b049-ea3cc6d16800 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:40.143064 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7bf64a-2b35-45d1-b049-ea3cc6d16800 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:40.156054 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7bf64a-2b35-45d1-b049-ea3cc6d16800 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:31:40.161446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-3e7bf64a-2b35-45d1-b049-ea3cc6d16800 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:31:40.161446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:40.161446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:40.161446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:40.161446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:40.161446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:40.161446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:40.161446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:40.161446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:40.161446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:40.161446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:40.161446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:40.161446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:31:40.161446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:31:40.161446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:31:40.161446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:31:40.163362 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:31:40.163362 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:31:40.163362 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:31:40.163362 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:31:40.163362 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:40.163362 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:40.163362 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:40.163362 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:40.163362 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:40.163362 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:40.163362 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:40.163362 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:40.163362 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:40.163362 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:40.163362 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:40.163362 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:40.164231 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:40.164231 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:40.164231 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:40.164231 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:40.164231 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:31:40.164231 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:31:40.164231 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:31:40.164231 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:31:40.164231 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:31:40.164231 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:31:40.164231 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:31:40.164231 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:40.164231 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 437/861] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 12:31:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:40.168970 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7bf64a-2b35-45d1-b049-ea3cc6d16800 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:40.228868 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6395db4e-b529-4d35-a321-771ebcdc4910 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:40.231767 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:40.272452 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064c5284-0f7d-4e4f-8693-dbd0ecd4ad04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:40.273209 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064c5284-0f7d-4e4f-8693-dbd0ecd4ad04 None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:40.273581 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064c5284-0f7d-4e4f-8693-dbd0ecd4ad04 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:40.273948 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064c5284-0f7d-4e4f-8693-dbd0ecd4ad04 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:40.274697 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064c5284-0f7d-4e4f-8693-dbd0ecd4ad04 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:40.276272 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064c5284-0f7d-4e4f-8693-dbd0ecd4ad04 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:40.313447 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7bf64a-2b35-45d1-b049-ea3cc6d16800 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:40.355477 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 425/862] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Oct 3 12:31:40 2022] POST /identity/v3/services => generated 251 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:31:40.382780 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b7880-ec08-408c-8038-3d8b38e60e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:40.383559 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b7880-ec08-408c-8038-3d8b38e60e4e None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:40.383844 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b7880-ec08-408c-8038-3d8b38e60e4e None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:40.384159 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b7880-ec08-408c-8038-3d8b38e60e4e None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:40.384836 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795b7880-ec08-408c-8038-3d8b38e60e4e None admin] RBAC: Adding query filter params (name=service) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:40.385142 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795b7880-ec08-408c-8038-3d8b38e60e4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:40.386563 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795b7880-ec08-408c-8038-3d8b38e60e4e None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:40.411745 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 438/863] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 12:31:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:31:40.419355 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064c5284-0f7d-4e4f-8693-dbd0ecd4ad04 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:40.544025 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:40.566804 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064c5284-0f7d-4e4f-8693-dbd0ecd4ad04 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:40.611104 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c9b5a-af7c-4d6d-b4b2-82b18b2a3da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:40.611718 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c9b5a-af7c-4d6d-b4b2-82b18b2a3da6 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:40.611992 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c9b5a-af7c-4d6d-b4b2-82b18b2a3da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:40.612256 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c9b5a-af7c-4d6d-b4b2-82b18b2a3da6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:40.627066 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0c9b5a-af7c-4d6d-b4b2-82b18b2a3da6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:40.628566 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0c9b5a-af7c-4d6d-b4b2-82b18b2a3da6 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:40.650385 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-2f0c9b5a-af7c-4d6d-b4b2-82b18b2a3da6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:40.650385 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:40.650385 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:40.650385 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:40.650385 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:40.650385 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:40.650385 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:40.650385 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:40.650385 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:40.650385 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:40.650385 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:40.650385 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:40.650385 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:40.650385 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:40.650385 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:40.650385 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:40.651770 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:40.651770 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:40.651770 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:40.651770 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:40.651770 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:40.651770 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:40.651770 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:40.651770 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:40.651770 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:40.651770 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:40.651770 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:40.651770 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:40.651770 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:40.651770 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:40.651770 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:40.651770 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:40.651770 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:40.653099 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:40.653099 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:40.653099 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:40.653099 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 426/864] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:40.657599 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b7880-ec08-408c-8038-3d8b38e60e4e None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:40.752053 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:40.775780 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b7880-ec08-408c-8038-3d8b38e60e4e None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:40.830688 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c330fcb5-f0a6-4a3c-a16f-d66f5ecceb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:40.832296 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330fcb5-f0a6-4a3c-a16f-d66f5ecceb7d None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:40.833197 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330fcb5-f0a6-4a3c-a16f-d66f5ecceb7d None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:40.834041 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330fcb5-f0a6-4a3c-a16f-d66f5ecceb7d None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:40.835062 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c330fcb5-f0a6-4a3c-a16f-d66f5ecceb7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:40.836040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c330fcb5-f0a6-4a3c-a16f-d66f5ecceb7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:40.838398 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c330fcb5-f0a6-4a3c-a16f-d66f5ecceb7d None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:40.858955 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 439/865] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:40.866961 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c9b5a-af7c-4d6d-b4b2-82b18b2a3da6 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:41.087832 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c9b5a-af7c-4d6d-b4b2-82b18b2a3da6 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:41.123462 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:41.128809 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969ca6a-8903-4f04-ad78-4afb5fe82678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:41.129453 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969ca6a-8903-4f04-ad78-4afb5fe82678 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:41.129727 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969ca6a-8903-4f04-ad78-4afb5fe82678 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:41.129995 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969ca6a-8903-4f04-ad78-4afb5fe82678 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:41.142276 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1969ca6a-8903-4f04-ad78-4afb5fe82678 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:41.143768 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1969ca6a-8903-4f04-ad78-4afb5fe82678 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:41.156026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-1969ca6a-8903-4f04-ad78-4afb5fe82678 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 12:31:41.156026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:41.156026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:41.156026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:41.156026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:41.156026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:41.156026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:41.156026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:41.156026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:41.156026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:41.156026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:41.156026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:41.156026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 12:31:41.156026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 12:31:41.156026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 12:31:41.156026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 12:31:41.156982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:41.156982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:41.156982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 12:31:41.156982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:41.156982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 12:31:41.156982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:41.156982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:41.156982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:41.156982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:41.156982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:41.156982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:41.156982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:41.156982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:41.156982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:41.156982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:41.156982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:41.156982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:41.156982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:41.157924 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:41.157924 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:41.157924 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 12:31:41.157924 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 12:31:41.157924 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 12:31:41.157924 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 12:31:41.157924 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 12:31:41.157924 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 12:31:41.157924 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 12:31:41.157924 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:41.157924 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 427/866] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Oct 3 12:31:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:41.163989 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c330fcb5-f0a6-4a3c-a16f-d66f5ecceb7d None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:41.271867 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c330fcb5-f0a6-4a3c-a16f-d66f5ecceb7d None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:41.307469 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b11e803-b3af-4066-8c72-1d1691a803e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:41.308080 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b11e803-b3af-4066-8c72-1d1691a803e6 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:41.308354 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b11e803-b3af-4066-8c72-1d1691a803e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:41.308622 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b11e803-b3af-4066-8c72-1d1691a803e6 None admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:41.309344 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b11e803-b3af-4066-8c72-1d1691a803e6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:41.309644 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b11e803-b3af-4066-8c72-1d1691a803e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:41.311112 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b11e803-b3af-4066-8c72-1d1691a803e6 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:41.325868 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:41.330315 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 440/867] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Oct 3 12:31:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:41.335876 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969ca6a-8903-4f04-ad78-4afb5fe82678 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:41.448916 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969ca6a-8903-4f04-ad78-4afb5fe82678 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:41.494091 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0642088-1192-4f31-9a00-a164f5fb68b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:41.494704 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0642088-1192-4f31-9a00-a164f5fb68b5 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:41.494983 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0642088-1192-4f31-9a00-a164f5fb68b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:41.495253 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0642088-1192-4f31-9a00-a164f5fb68b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:41.561024 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:41.570793 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0642088-1192-4f31-9a00-a164f5fb68b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:41.572273 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0642088-1192-4f31-9a00-a164f5fb68b5 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:41.581403 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7412a5-4eb2-4c5f-9f4f-cbe43d4289b8 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:41.581978 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7412a5-4eb2-4c5f-9f4f-cbe43d4289b8 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:41.582474 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7412a5-4eb2-4c5f-9f4f-cbe43d4289b8 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:41.584867 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 441/868] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:41.590309 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a2ce3-02bc-4a51-9af1-561db81c2fc3 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:41.590849 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a2ce3-02bc-4a51-9af1-561db81c2fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:41.591353 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a2ce3-02bc-4a51-9af1-561db81c2fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:41.619457 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-a06a2ce3-02bc-4a51-9af1-561db81c2fc3 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:41.621332 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-a06a2ce3-02bc-4a51-9af1-561db81c2fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:41.682902 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 428/869] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:31:41 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 12:31:41.688465 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0642088-1192-4f31-9a00-a164f5fb68b5 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:41.723627 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 442/870] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:41 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:31:41.797713 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:41.807109 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0642088-1192-4f31-9a00-a164f5fb68b5 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:41.842469 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac93e2-6064-4664-9841-77912a0d0698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:41.843384 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac93e2-6064-4664-9841-77912a0d0698 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:41.843665 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac93e2-6064-4664-9841-77912a0d0698 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:41.843939 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac93e2-6064-4664-9841-77912a0d0698 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:41.854700 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ac93e2-6064-4664-9841-77912a0d0698 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:41.856164 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ac93e2-6064-4664-9841-77912a0d0698 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:41.871855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-10ac93e2-6064-4664-9841-77912a0d0698 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:41.871855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:41.871855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:41.871855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:41.871855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:41.871855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:41.871855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:41.871855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:41.871855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:41.871855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:41.871855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:41.871855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:41.871855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:41.871855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:41.871855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:41.871855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:41.872902 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:41.872902 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:41.872902 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:41.872902 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:41.872902 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:41.872902 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:41.872902 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:41.872902 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:41.872902 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:41.872902 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:41.872902 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:41.872902 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:41.872902 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:41.872902 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:41.872902 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:41.872902 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:41.872902 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:41.873876 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:41.873876 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:41.873876 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:41.873876 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 429/871] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:31:41.878852 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a2ce3-02bc-4a51-9af1-561db81c2fc3 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:41.990834 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a2ce3-02bc-4a51-9af1-561db81c2fc3 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:42.022862 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:42.029426 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac241af1-2968-4b29-ae2d-106f2af65e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:42.030317 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac241af1-2968-4b29-ae2d-106f2af65e0b None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:42.030813 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac241af1-2968-4b29-ae2d-106f2af65e0b None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:42.031343 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac241af1-2968-4b29-ae2d-106f2af65e0b None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:42.032217 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac241af1-2968-4b29-ae2d-106f2af65e0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:42.032747 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac241af1-2968-4b29-ae2d-106f2af65e0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:42.034316 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac241af1-2968-4b29-ae2d-106f2af65e0b None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:42.039276 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac93e2-6064-4664-9841-77912a0d0698 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:42.051380 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 443/872] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:42.054783 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac241af1-2968-4b29-ae2d-106f2af65e0b None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:42.162594 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac241af1-2968-4b29-ae2d-106f2af65e0b None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:42.164501 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac93e2-6064-4664-9841-77912a0d0698 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:42.196139 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13086264-39d8-4d80-ba99-49baee9d5934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:42.196775 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13086264-39d8-4d80-ba99-49baee9d5934 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:42.197056 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13086264-39d8-4d80-ba99-49baee9d5934 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:42.197331 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13086264-39d8-4d80-ba99-49baee9d5934 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:42.201806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087c540-2a2a-4ea7-bf66-5693ac85e20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:42.202434 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087c540-2a2a-4ea7-bf66-5693ac85e20d None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:42.202717 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087c540-2a2a-4ea7-bf66-5693ac85e20d None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:42.203012 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087c540-2a2a-4ea7-bf66-5693ac85e20d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:42.203592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f087c540-2a2a-4ea7-bf66-5693ac85e20d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:42.203906 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13086264-39d8-4d80-ba99-49baee9d5934 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:42.204948 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f087c540-2a2a-4ea7-bf66-5693ac85e20d None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:42.205321 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13086264-39d8-4d80-ba99-49baee9d5934 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:42.214017 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-f087c540-2a2a-4ea7-bf66-5693ac85e20d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 12:31:42.214017 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:42.214017 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 12:31:42.214017 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 12:31:42.214017 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 12:31:42.214017 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 12:31:42.214017 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 12:31:42.214017 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:31:42.214017 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 12:31:42.214017 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:42.214017 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 12:31:42.214017 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:42.214017 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:42.214017 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:42.214017 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:42.214017 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:42.214017 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:42.215288 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:42.215288 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:42.215288 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:42.215288 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:42.215288 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:42.215288 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:42.215288 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 12:31:42.215288 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 12:31:42.215288 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 12:31:42.215288 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 12:31:42.215288 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:42.215288 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:42.215288 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:42.215288 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:42.215288 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:42.215288 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:42.216436 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:42.216436 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:42.216436 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:42.216436 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:42.216436 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:42.216436 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:42.216436 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:42.216436 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:42.216436 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:42.216436 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:42.216436 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 12:31:42.216436 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:31:42.216436 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 12:31:42.216436 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:42.217584 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-13086264-39d8-4d80-ba99-49baee9d5934 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:42.217584 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:42.217584 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:42.217584 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:42.217584 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:42.217584 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:42.217584 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:42.217584 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:42.217584 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:42.217584 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:42.217584 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:42.217584 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:42.217584 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 12:31:42.217584 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 12:31:42.217584 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 12:31:42.218883 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 12:31:42.218883 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:42.218883 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:42.218883 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:42.218883 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:42.218883 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:42.218883 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:42.218883 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:42.218883 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:42.218883 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:42.218883 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:42.218883 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:42.218883 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:42.218883 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:42.218883 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:42.218883 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:42.218883 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:42.220042 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 12:31:42.220042 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 12:31:42.220042 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 12:31:42.220042 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 12:31:42.220042 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 12:31:42.220042 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 12:31:42.220042 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:42.220042 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:42.223707 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 444/873] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:31:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:42.226095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e17eec6-67ed-4eea-980e-65120b4eca75 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:42.226387 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e17eec6-67ed-4eea-980e-65120b4eca75 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:42.226791 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e17eec6-67ed-4eea-980e-65120b4eca75 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:42.227051 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 430/874] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Oct 3 12:31:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:42.227926 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087c540-2a2a-4ea7-bf66-5693ac85e20d None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:42.231624 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 445/875] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:42.231624 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e17eec6-67ed-4eea-980e-65120b4eca75 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:42.251015 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:42.288244 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:42.341655 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e17eec6-67ed-4eea-980e-65120b4eca75 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:42.341885 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087c540-2a2a-4ea7-bf66-5693ac85e20d None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:42.376934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e786c-a09d-4ef5-babb-ef3bfbce6066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:42.377557 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e786c-a09d-4ef5-babb-ef3bfbce6066 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:42.377839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e786c-a09d-4ef5-babb-ef3bfbce6066 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:42.378110 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e786c-a09d-4ef5-babb-ef3bfbce6066 None admin] PATH_INFO: `/v3/services` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:42.378736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23e786c-a09d-4ef5-babb-ef3bfbce6066 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:42.378946 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1b9e9b-a8aa-4b43-9ec5-c7fd9c7f0ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:42.379074 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23e786c-a09d-4ef5-babb-ef3bfbce6066 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:42.379625 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b9e9b-a8aa-4b43-9ec5-c7fd9c7f0ada None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:42.379909 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b9e9b-a8aa-4b43-9ec5-c7fd9c7f0ada None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:42.380175 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b9e9b-a8aa-4b43-9ec5-c7fd9c7f0ada None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:42.380498 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23e786c-a09d-4ef5-babb-ef3bfbce6066 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:42.380929 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1b9e9b-a8aa-4b43-9ec5-c7fd9c7f0ada None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:42.381231 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1b9e9b-a8aa-4b43-9ec5-c7fd9c7f0ada None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:42.382740 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1b9e9b-a8aa-4b43-9ec5-c7fd9c7f0ada None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:42.386144 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 446/876] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Oct 3 12:31:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:42.387284 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47844695-7417-44af-9794-361b608775e1 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:42.387639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47844695-7417-44af-9794-361b608775e1 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:42.387885 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47844695-7417-44af-9794-361b608775e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:42.404540 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 431/877] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 12:31:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:42.404540 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1b9e9b-a8aa-4b43-9ec5-c7fd9c7f0ada None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:42.410671 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-47844695-7417-44af-9794-361b608775e1 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:42.411960 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-47844695-7417-44af-9794-361b608775e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:42.479605 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:42.499457 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 447/878] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:42 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:31:42.500263 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47844695-7417-44af-9794-361b608775e1 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:42.523657 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1b9e9b-a8aa-4b43-9ec5-c7fd9c7f0ada None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:42.551774 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:42.560650 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc8691-d698-4c93-944e-1ce356f7a574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:42.568949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc8691-d698-4c93-944e-1ce356f7a574 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:42.568949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc8691-d698-4c93-944e-1ce356f7a574 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:42.568949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc8691-d698-4c93-944e-1ce356f7a574 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:42.568949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fc8691-d698-4c93-944e-1ce356f7a574 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:42.568949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fc8691-d698-4c93-944e-1ce356f7a574 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:42.568949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fc8691-d698-4c93-944e-1ce356f7a574 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:42.575789 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 432/879] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Oct 3 12:31:42 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:42.612361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47844695-7417-44af-9794-361b608775e1 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:42.651700 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642da5a-0bb5-40c7-99b3-d3d09d71df9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:42.652327 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642da5a-0bb5-40c7-99b3-d3d09d71df9c None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:42.652604 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642da5a-0bb5-40c7-99b3-d3d09d71df9c None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:42.652872 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642da5a-0bb5-40c7-99b3-d3d09d71df9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:42.653607 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d642da5a-0bb5-40c7-99b3-d3d09d71df9c None admin] RBAC: Adding query filter params (role.id=ff6b2d4413644db39a2bd8576afc6ccb, scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7, user.id=2e6e4387a7c24bbc82c22eefe2a3d138) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:42.653913 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d642da5a-0bb5-40c7-99b3-d3d09d71df9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:42.655474 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d642da5a-0bb5-40c7-99b3-d3d09d71df9c None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:42.662759 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 448/880] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:31:42 2022] GET /identity/v3/role_assignments?user.id=2e6e4387a7c24bbc82c22eefe2a3d138&scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7&role.id=ff6b2d4413644db39a2bd8576afc6ccb => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:42.783189 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc8691-d698-4c93-944e-1ce356f7a574 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:42.857481 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e1cf6-10fe-4a77-9ad2-5f8c4a36b9fc None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:42.858099 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e1cf6-10fe-4a77-9ad2-5f8c4a36b9fc None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:42.858598 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e1cf6-10fe-4a77-9ad2-5f8c4a36b9fc None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:42.861292 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 449/881] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 12:31:42.866393 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4c393-363f-4600-9a5d-eba820c5bdf0 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:42.866393 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4c393-363f-4600-9a5d-eba820c5bdf0 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:42.866699 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4c393-363f-4600-9a5d-eba820c5bdf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:42.890521 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-ecf4c393-363f-4600-9a5d-eba820c5bdf0 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:42.891150 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-ecf4c393-363f-4600-9a5d-eba820c5bdf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:42.906648 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc8691-d698-4c93-944e-1ce356f7a574 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:42.942022 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:42.951345 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa10b73-677a-452e-8e0d-57b1a2febf23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:42.951988 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa10b73-677a-452e-8e0d-57b1a2febf23 None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:42.952258 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa10b73-677a-452e-8e0d-57b1a2febf23 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:42.952522 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa10b73-677a-452e-8e0d-57b1a2febf23 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:42.953183 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa10b73-677a-452e-8e0d-57b1a2febf23 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:42.954537 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa10b73-677a-452e-8e0d-57b1a2febf23 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:42.976184 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 450/882] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:42 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 12:31:43.041019 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 433/883] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Oct 3 12:31:42 2022] POST /identity/v3/services => generated 269 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:31:43.264637 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4c393-363f-4600-9a5d-eba820c5bdf0 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:43.307078 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:43.370022 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4c393-363f-4600-9a5d-eba820c5bdf0 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:43.416391 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed69e1-9397-4514-9524-d2343ae2293e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:43.417042 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed69e1-9397-4514-9524-d2343ae2293e None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:43.417321 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed69e1-9397-4514-9524-d2343ae2293e None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:43.417585 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed69e1-9397-4514-9524-d2343ae2293e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:43.418159 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed69e1-9397-4514-9524-d2343ae2293e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:43.419550 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed69e1-9397-4514-9524-d2343ae2293e None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:43.429523 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-3aed69e1-9397-4514-9524-d2343ae2293e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 03 12:31:43.429523 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:43.429523 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 12:31:43.429523 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 12:31:43.429523 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 12:31:43.429523 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 12:31:43.429523 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 12:31:43.429523 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:31:43.429523 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 03 12:31:43.429523 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:43.429523 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 12:31:43.429523 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:43.429523 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:43.429523 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:43.429523 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:43.429523 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:43.429523 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:43.430596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:43.430596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:43.430596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:43.430596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:43.430596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:43.430596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:43.430596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 12:31:43.430596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 12:31:43.430596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 12:31:43.430596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 12:31:43.430596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:43.430596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:43.430596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:43.430596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:43.430596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:43.430596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:43.431462 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:43.431462 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:43.431462 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:43.431462 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:43.431462 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:43.431462 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:43.431462 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:43.431462 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:43.431462 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:43.431462 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:43.431462 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 12:31:43.431462 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:31:43.431462 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 03 12:31:43.431462 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:43.431462 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 451/884] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 12:31:43 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:43.436604 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa10b73-677a-452e-8e0d-57b1a2febf23 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:43.546485 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa10b73-677a-452e-8e0d-57b1a2febf23 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:43.582174 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f947ed-263e-4adb-a29b-99f9e1250c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:43.583916 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f947ed-263e-4adb-a29b-99f9e1250c3f None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:43.584197 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f947ed-263e-4adb-a29b-99f9e1250c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:43.584469 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f947ed-263e-4adb-a29b-99f9e1250c3f None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:43.585113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f947ed-263e-4adb-a29b-99f9e1250c3f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:43.585411 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f947ed-263e-4adb-a29b-99f9e1250c3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:43.586809 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f947ed-263e-4adb-a29b-99f9e1250c3f None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:43.592558 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 434/885] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 12:31:43 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:43.598525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed69e1-9397-4514-9524-d2343ae2293e None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:43.701965 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed69e1-9397-4514-9524-d2343ae2293e None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:43.718509 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:43.737170 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b4ef5-34ea-4a11-94d1-fac3827341ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:43.743748 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b4ef5-34ea-4a11-94d1-fac3827341ed None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:43.743748 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b4ef5-34ea-4a11-94d1-fac3827341ed None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:43.743748 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b4ef5-34ea-4a11-94d1-fac3827341ed None admin] PATH_INFO: `/v3/services` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:43.743748 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62b4ef5-34ea-4a11-94d1-fac3827341ed None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:43.743748 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62b4ef5-34ea-4a11-94d1-fac3827341ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:43.743748 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62b4ef5-34ea-4a11-94d1-fac3827341ed None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:43.747743 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 452/886] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 12:31:43 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:43.754240 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f947ed-263e-4adb-a29b-99f9e1250c3f None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:43.814000 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:43.859182 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f947ed-263e-4adb-a29b-99f9e1250c3f None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:43.894054 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef534da-6368-434d-9f62-cf688402c841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:43.894661 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef534da-6368-434d-9f62-cf688402c841 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:43.894945 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef534da-6368-434d-9f62-cf688402c841 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:43.895202 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef534da-6368-434d-9f62-cf688402c841 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:43.895934 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef534da-6368-434d-9f62-cf688402c841 None admin] RBAC: Adding query filter params (interface=public, service_id=c0bb2f832a504a2c89bfc51b5703ef36, region_id=RegionOne) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:43.896838 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef534da-6368-434d-9f62-cf688402c841 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:43.898319 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef534da-6368-434d-9f62-cf688402c841 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:43.904213 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 435/887] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Oct 3 12:31:43 2022] GET /identity/v3/endpoints?service_id=c0bb2f832a504a2c89bfc51b5703ef36&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:43.909777 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b4ef5-34ea-4a11-94d1-fac3827341ed None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:44.017072 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b4ef5-34ea-4a11-94d1-fac3827341ed None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:44.052259 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af025f8-9551-4278-a29b-df7717f57b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:44.053682 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af025f8-9551-4278-a29b-df7717f57b98 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:44.054013 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af025f8-9551-4278-a29b-df7717f57b98 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:44.054235 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af025f8-9551-4278-a29b-df7717f57b98 None admin] PATH_INFO: `/v3/services` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:44.054839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af025f8-9551-4278-a29b-df7717f57b98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:44.056218 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af025f8-9551-4278-a29b-df7717f57b98 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:44.062388 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 453/888] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Oct 3 12:31:43 2022] GET /identity/v3/services => generated 1268 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:31:44.939607 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c76d8d-942c-4733-9f72-3f977c914297 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:44.940139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c76d8d-942c-4733-9f72-3f977c914297 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:44.940139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c76d8d-942c-4733-9f72-3f977c914297 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:44.941933 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 436/889] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:44.948591 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f19a71f-5a00-4941-bdb9-3f0a92bd984e None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:44.949598 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f19a71f-5a00-4941-bdb9-3f0a92bd984e None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:44.950081 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f19a71f-5a00-4941-bdb9-3f0a92bd984e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:44.970506 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-8f19a71f-5a00-4941-bdb9-3f0a92bd984e None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:44.976498 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-8f19a71f-5a00-4941-bdb9-3f0a92bd984e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:44.995594 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:45.044161 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 454/890] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:44 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:31:45.048959 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e84aa-c7c7-49a5-875e-f7d50ac7f123 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:45.048959 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e84aa-c7c7-49a5-875e-f7d50ac7f123 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:45.048959 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e84aa-c7c7-49a5-875e-f7d50ac7f123 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:45.049518 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 437/891] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:45.056002 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14865816-2c01-4863-b663-157257959e59 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:45.056474 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14865816-2c01-4863-b663-157257959e59 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:45.056891 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14865816-2c01-4863-b663-157257959e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:45.077214 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-14865816-2c01-4863-b663-157257959e59 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:45.078649 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-14865816-2c01-4863-b663-157257959e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:45.157158 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 455/892] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:45 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:31:45.334487 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e84aa-c7c7-49a5-875e-f7d50ac7f123 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:45.355759 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d02b379-bef4-45f7-9722-3975363ac723 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:45.356415 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d02b379-bef4-45f7-9722-3975363ac723 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:45.356898 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d02b379-bef4-45f7-9722-3975363ac723 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:45.358873 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 456/893] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 12:31:45.364638 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe12f7-4ef5-4b8f-8f7c-bab4ce0bc9e8 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:45.365752 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe12f7-4ef5-4b8f-8f7c-bab4ce0bc9e8 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:45.366233 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe12f7-4ef5-4b8f-8f7c-bab4ce0bc9e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:45.389176 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-8afe12f7-4ef5-4b8f-8f7c-bab4ce0bc9e8 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:45.390356 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-8afe12f7-4ef5-4b8f-8f7c-bab4ce0bc9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:45.459350 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e84aa-c7c7-49a5-875e-f7d50ac7f123 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:45.475101 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 457/894] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:45 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 12:31:45.475101 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe12f7-4ef5-4b8f-8f7c-bab4ce0bc9e8 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:45.480891 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:45.494878 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:45.506458 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934f730-b906-49f7-9353-64bbe704c802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:45.507010 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934f730-b906-49f7-9353-64bbe704c802 None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:45.507285 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934f730-b906-49f7-9353-64bbe704c802 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:45.507591 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934f730-b906-49f7-9353-64bbe704c802 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:45.508232 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3934f730-b906-49f7-9353-64bbe704c802 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:45.509577 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3934f730-b906-49f7-9353-64bbe704c802 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:45.588118 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe12f7-4ef5-4b8f-8f7c-bab4ce0bc9e8 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:45.619747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 438/895] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Oct 3 12:31:45 2022] POST /identity/v3/services => generated 241 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:31:45.637951 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da7b0aa-2ad7-430a-a8be-1bd960fb923f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:45.638704 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da7b0aa-2ad7-430a-a8be-1bd960fb923f None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:45.638990 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da7b0aa-2ad7-430a-a8be-1bd960fb923f None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:45.639262 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da7b0aa-2ad7-430a-a8be-1bd960fb923f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:45.653452 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da7b0aa-2ad7-430a-a8be-1bd960fb923f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:31:45.658642 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-2da7b0aa-2ad7-430a-a8be-1bd960fb923f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 12:31:45.658642 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:45.658642 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:45.658642 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:45.658642 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:45.658642 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:45.658642 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:45.658642 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:45.658642 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:45.658642 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:45.658642 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:45.658642 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:45.658642 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:31:45.658642 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:31:45.658642 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:31:45.658642 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:31:45.659648 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:31:45.659648 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:31:45.659648 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:31:45.659648 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:31:45.659648 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:45.659648 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:45.659648 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:45.659648 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:45.659648 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:45.659648 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:45.659648 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:45.659648 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:45.659648 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:45.659648 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:45.659648 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:45.659648 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:45.660469 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:45.660469 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:45.660469 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:45.660469 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:45.660469 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:31:45.660469 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:31:45.660469 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:31:45.660469 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:31:45.660469 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:31:45.660469 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:31:45.660469 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 12:31:45.660469 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:45.665402 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 458/896] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Oct 3 12:31:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:45.667083 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934f730-b906-49f7-9353-64bbe704c802 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:45.740195 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da7b0aa-2ad7-430a-a8be-1bd960fb923f None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:45.785840 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934f730-b906-49f7-9353-64bbe704c802 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:45.829050 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb7fce2-a80b-4b87-81ad-aecec57a0f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:45.829935 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7fce2-a80b-4b87-81ad-aecec57a0f4e None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:45.830357 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7fce2-a80b-4b87-81ad-aecec57a0f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:45.830779 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7fce2-a80b-4b87-81ad-aecec57a0f4e None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:45.831474 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb7fce2-a80b-4b87-81ad-aecec57a0f4e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:45.831935 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb7fce2-a80b-4b87-81ad-aecec57a0f4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:45.833453 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb7fce2-a80b-4b87-81ad-aecec57a0f4e None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:45.839691 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:45.843606 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 439/897] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Oct 3 12:31:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:45.850046 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb7fce2-a80b-4b87-81ad-aecec57a0f4e None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:45.879882 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da7b0aa-2ad7-430a-a8be-1bd960fb923f None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:45.912911 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:45.920991 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6959686e-9549-4c4c-b75e-28844c1f1fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:45.921990 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959686e-9549-4c4c-b75e-28844c1f1fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:45.922919 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959686e-9549-4c4c-b75e-28844c1f1fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:45.923368 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959686e-9549-4c4c-b75e-28844c1f1fd6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:45.924928 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6959686e-9549-4c4c-b75e-28844c1f1fd6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:45.927559 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6959686e-9549-4c4c-b75e-28844c1f1fd6 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:45.938119 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-6959686e-9549-4c4c-b75e-28844c1f1fd6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 03 12:31:45.938119 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:45.938119 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 12:31:45.938119 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 12:31:45.938119 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 12:31:45.938119 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 12:31:45.938119 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 12:31:45.938119 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:31:45.938119 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 03 12:31:45.938119 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:45.938119 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 12:31:45.938119 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:45.938119 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:45.938119 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:45.938119 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:45.938119 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:45.939409 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:45.939409 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:45.939409 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:45.939409 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:45.939409 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:45.939409 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:45.939409 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:45.939409 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 12:31:45.939409 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 12:31:45.939409 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 12:31:45.939409 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 12:31:45.939409 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:45.939409 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:45.939409 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:45.939409 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:45.939409 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:45.939409 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:45.940501 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:45.940501 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:45.940501 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:45.940501 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:45.940501 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:45.940501 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:45.940501 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:45.940501 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:45.940501 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:45.940501 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:45.940501 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 12:31:45.940501 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:31:45.940501 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 03 12:31:45.940501 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:45.944196 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 459/898] 10.222.0.33 () {56 vars in 1132 bytes} [Mon Oct 3 12:31:45 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:31:45.949662 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6959686e-9549-4c4c-b75e-28844c1f1fd6 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:45.970130 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb7fce2-a80b-4b87-81ad-aecec57a0f4e None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:46.018135 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95cfda8-a514-4021-baf6-b1e47618276d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:46.018975 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95cfda8-a514-4021-baf6-b1e47618276d None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:46.019377 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95cfda8-a514-4021-baf6-b1e47618276d None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:46.019659 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95cfda8-a514-4021-baf6-b1e47618276d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:46.043095 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95cfda8-a514-4021-baf6-b1e47618276d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:46.044651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95cfda8-a514-4021-baf6-b1e47618276d None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:46.064620 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-a95cfda8-a514-4021-baf6-b1e47618276d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:46.064620 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:46.064620 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:46.064620 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:46.064620 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:46.064620 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:46.064620 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:46.064620 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:46.064620 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:46.064620 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:46.064620 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:46.064620 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:46.064620 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:46.064620 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:46.064620 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:46.064620 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:46.065638 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:46.065638 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:46.065638 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:46.065638 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:46.065638 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:46.065638 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:46.065638 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:46.065638 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:46.065638 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:46.065638 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:46.065638 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:46.065638 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:46.065638 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:46.065638 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:46.065638 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:46.065638 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:46.065638 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:46.066538 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:46.066538 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:46.066538 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:46.066538 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 440/899] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:31:46.071634 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95cfda8-a514-4021-baf6-b1e47618276d None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:46.078670 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6959686e-9549-4c4c-b75e-28844c1f1fd6 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:46.117918 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a53e13-12da-4593-b067-35c995923b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:46.119006 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a53e13-12da-4593-b067-35c995923b71 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:46.120911 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a53e13-12da-4593-b067-35c995923b71 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:46.121418 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a53e13-12da-4593-b067-35c995923b71 None admin] PATH_INFO: `/v3/services` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:46.122245 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a53e13-12da-4593-b067-35c995923b71 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:46.122802 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a53e13-12da-4593-b067-35c995923b71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:46.124634 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a53e13-12da-4593-b067-35c995923b71 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:46.126972 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:46.137190 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 460/900] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Oct 3 12:31:45 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:46.138545 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a53e13-12da-4593-b067-35c995923b71 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:46.181835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:46.189449 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95cfda8-a514-4021-baf6-b1e47618276d None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:46.248243 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c9d2c7-79de-471a-9807-c581a170452c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:46.250165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9d2c7-79de-471a-9807-c581a170452c None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:46.250439 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9d2c7-79de-471a-9807-c581a170452c None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:46.250707 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9d2c7-79de-471a-9807-c581a170452c None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:46.251408 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c9d2c7-79de-471a-9807-c581a170452c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:46.251712 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c9d2c7-79de-471a-9807-c581a170452c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:46.253071 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c9d2c7-79de-471a-9807-c581a170452c None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:46.267401 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 441/901] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:31:46.272349 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c9d2c7-79de-471a-9807-c581a170452c None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:46.274816 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a53e13-12da-4593-b067-35c995923b71 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:46.316802 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1cb0c2-a0c1-4df0-a447-9e990c1477a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:46.317618 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1cb0c2-a0c1-4df0-a447-9e990c1477a9 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:46.318546 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1cb0c2-a0c1-4df0-a447-9e990c1477a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:46.319538 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1cb0c2-a0c1-4df0-a447-9e990c1477a9 None admin] PATH_INFO: `/v3/services` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:46.320326 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1cb0c2-a0c1-4df0-a447-9e990c1477a9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:46.320738 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1cb0c2-a0c1-4df0-a447-9e990c1477a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:46.323402 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1cb0c2-a0c1-4df0-a447-9e990c1477a9 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:46.330229 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 461/902] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Oct 3 12:31:46 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:46.336754 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1cb0c2-a0c1-4df0-a447-9e990c1477a9 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:46.395844 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c9d2c7-79de-471a-9807-c581a170452c None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:46.404424 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:46.432494 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8db4f-025d-400a-b95d-3ea9f3569ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:46.433137 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8db4f-025d-400a-b95d-3ea9f3569ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:46.433449 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8db4f-025d-400a-b95d-3ea9f3569ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:46.433719 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8db4f-025d-400a-b95d-3ea9f3569ff0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:46.442085 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc8db4f-025d-400a-b95d-3ea9f3569ff0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:46.444387 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc8db4f-025d-400a-b95d-3ea9f3569ff0 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:46.445325 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1cb0c2-a0c1-4df0-a447-9e990c1477a9 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:46.455907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-9bc8db4f-025d-400a-b95d-3ea9f3569ff0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 12:31:46.455907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:46.455907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:46.455907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:46.455907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:46.455907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:46.455907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:46.455907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:46.455907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:46.455907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:46.455907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:46.455907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:46.455907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 12:31:46.455907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 12:31:46.455907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 12:31:46.455907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 12:31:46.456882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:46.456882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:46.456882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 12:31:46.456882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:46.456882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 12:31:46.456882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:46.456882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:46.456882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:46.456882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:46.456882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:46.456882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:46.456882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:46.456882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:46.456882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:46.456882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:46.456882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:46.456882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:46.456882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:46.457818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:46.457818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:46.457818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 12:31:46.457818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 12:31:46.457818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 12:31:46.457818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 12:31:46.457818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 12:31:46.457818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 12:31:46.457818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 12:31:46.457818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:46.457818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 442/903] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Oct 3 12:31:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:46.458851 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d50a6-63ff-4082-905e-2f3fa972adbe None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:46.458851 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d50a6-63ff-4082-905e-2f3fa972adbe None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:46.458851 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d50a6-63ff-4082-905e-2f3fa972adbe None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:46.464328 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 443/904] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:46.465088 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d50a6-63ff-4082-905e-2f3fa972adbe None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:46.482932 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:46.487479 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8109a87-ad23-4d79-bd8f-c27df3d9d34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:46.488138 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8109a87-ad23-4d79-bd8f-c27df3d9d34f None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:46.488422 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8109a87-ad23-4d79-bd8f-c27df3d9d34f None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:46.488692 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8109a87-ad23-4d79-bd8f-c27df3d9d34f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:46.489647 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8109a87-ad23-4d79-bd8f-c27df3d9d34f None admin] RBAC: Adding query filter params (interface=public, service_id=d31e058b5e2b4686a9453a360abdbe85, region_id=RegionOne) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:46.489755 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8109a87-ad23-4d79-bd8f-c27df3d9d34f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:46.491308 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8109a87-ad23-4d79-bd8f-c27df3d9d34f None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:46.496920 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 462/905] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Oct 3 12:31:46 2022] GET /identity/v3/endpoints?service_id=d31e058b5e2b4686a9453a360abdbe85&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:46.498374 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1df34-23a7-426b-827b-b34a39ace82b None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:46.498651 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1df34-23a7-426b-827b-b34a39ace82b None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:46.498923 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1df34-23a7-426b-827b-b34a39ace82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:46.517909 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-a9b1df34-23a7-426b-827b-b34a39ace82b None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:46.519080 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-a9b1df34-23a7-426b-827b-b34a39ace82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:46.576700 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d50a6-63ff-4082-905e-2f3fa972adbe None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:46.592154 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 463/906] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:31:46.604205 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b1df34-23a7-426b-827b-b34a39ace82b None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:46.612841 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba23cfc-5c67-40f7-ba2c-df6a67e4c4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:46.613448 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba23cfc-5c67-40f7-ba2c-df6a67e4c4a0 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:46.613722 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba23cfc-5c67-40f7-ba2c-df6a67e4c4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:46.613991 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba23cfc-5c67-40f7-ba2c-df6a67e4c4a0 None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:46.614703 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba23cfc-5c67-40f7-ba2c-df6a67e4c4a0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:46.615111 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba23cfc-5c67-40f7-ba2c-df6a67e4c4a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:46.617102 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba23cfc-5c67-40f7-ba2c-df6a67e4c4a0 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:46.642411 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 444/907] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Oct 3 12:31:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:46.643680 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba23cfc-5c67-40f7-ba2c-df6a67e4c4a0 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:46.682641 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:46.718823 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:46.739766 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b1df34-23a7-426b-827b-b34a39ace82b None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:46.750589 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba23cfc-5c67-40f7-ba2c-df6a67e4c4a0 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:46.788605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb01e9e-1dcb-4509-bdd9-0b8cb972f316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:46.789214 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb01e9e-1dcb-4509-bdd9-0b8cb972f316 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:46.789489 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb01e9e-1dcb-4509-bdd9-0b8cb972f316 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:46.789757 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb01e9e-1dcb-4509-bdd9-0b8cb972f316 None admin] PATH_INFO: `/v3/services` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:46.790358 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb01e9e-1dcb-4509-bdd9-0b8cb972f316 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:46.791723 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb01e9e-1dcb-4509-bdd9-0b8cb972f316 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:46.794045 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b1d1a-a708-4ca4-8268-9db6368d4eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:46.795165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b1d1a-a708-4ca4-8268-9db6368d4eb7 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:46.795693 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b1d1a-a708-4ca4-8268-9db6368d4eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:46.796171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b1d1a-a708-4ca4-8268-9db6368d4eb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:46.803357 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 464/908] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Oct 3 12:31:46 2022] GET /identity/v3/services => generated 1497 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:31:46.875820 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb01e9e-1dcb-4509-bdd9-0b8cb972f316 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:46.878822 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5b1d1a-a708-4ca4-8268-9db6368d4eb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:46.880347 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5b1d1a-a708-4ca4-8268-9db6368d4eb7 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:46.973676 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:46.994967 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb01e9e-1dcb-4509-bdd9-0b8cb972f316 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:47.015603 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 445/909] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:31:46 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 12:31:47.021146 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b1d1a-a708-4ca4-8268-9db6368d4eb7 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:47.029583 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:47.046703 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f6ba3-b23e-4eba-93fb-4f483fb73127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:47.047714 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f6ba3-b23e-4eba-93fb-4f483fb73127 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:47.047714 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f6ba3-b23e-4eba-93fb-4f483fb73127 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:47.047980 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f6ba3-b23e-4eba-93fb-4f483fb73127 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:47.048565 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f6ba3-b23e-4eba-93fb-4f483fb73127 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:47.052359 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f6ba3-b23e-4eba-93fb-4f483fb73127 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:47.061639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-661f6ba3-b23e-4eba-93fb-4f483fb73127 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 03 12:31:47.061639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:47.061639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 12:31:47.061639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 12:31:47.061639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 12:31:47.061639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 12:31:47.061639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 12:31:47.061639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:31:47.061639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 03 12:31:47.061639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:47.061639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 12:31:47.061639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:47.061639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:47.061639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:47.061639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:47.061639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:47.061639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:47.063053 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:47.063053 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:47.063053 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:47.063053 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:47.063053 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:47.063053 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:47.063053 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 12:31:47.063053 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 12:31:47.063053 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 12:31:47.063053 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 12:31:47.063053 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:47.063053 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:47.063053 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:47.063053 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:47.063053 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:47.063053 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:47.064282 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:47.064282 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:47.064282 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:47.064282 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:47.064282 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:47.064282 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:47.064282 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:47.064282 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:47.064282 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:47.064282 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:47.064282 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 12:31:47.064282 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:31:47.064282 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 03 12:31:47.064282 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:47.064282 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 465/910] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 12:31:46 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:31:47.068330 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f6ba3-b23e-4eba-93fb-4f483fb73127 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:47.141497 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b1d1a-a708-4ca4-8268-9db6368d4eb7 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:47.192025 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca2b2ec-8ed5-4a79-8089-bb11c8320b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:47.193701 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca2b2ec-8ed5-4a79-8089-bb11c8320b78 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:47.194892 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca2b2ec-8ed5-4a79-8089-bb11c8320b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:47.195394 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca2b2ec-8ed5-4a79-8089-bb11c8320b78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:47.198401 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f6ba3-b23e-4eba-93fb-4f483fb73127 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:47.209560 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca2b2ec-8ed5-4a79-8089-bb11c8320b78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:47.211279 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca2b2ec-8ed5-4a79-8089-bb11c8320b78 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:47.232455 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-dca2b2ec-8ed5-4a79-8089-bb11c8320b78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:47.232455 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:47.232455 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:47.232455 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:47.232455 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:47.232455 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:47.232455 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:47.232455 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:47.232455 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:47.232455 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:47.232455 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:47.232455 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:47.232455 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:47.232455 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:47.232455 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:47.232455 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:47.233556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:47.233556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:47.233556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:47.233556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:47.233556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:47.233556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:47.233556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:47.233556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:47.233556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:47.233556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:47.233556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:47.233556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:47.233556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:47.233556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:47.233556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:47.233556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:47.233556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:47.234433 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:47.234433 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:47.234433 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:47.241330 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 446/911] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:31:47.243261 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca2b2ec-8ed5-4a79-8089-bb11c8320b78 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:47.254264 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3137605-aa55-4872-81c7-adbfe370707a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:47.254991 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3137605-aa55-4872-81c7-adbfe370707a None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:47.256087 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3137605-aa55-4872-81c7-adbfe370707a None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:47.256558 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3137605-aa55-4872-81c7-adbfe370707a None admin] PATH_INFO: `/v3/services` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:47.257358 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3137605-aa55-4872-81c7-adbfe370707a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:47.257766 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3137605-aa55-4872-81c7-adbfe370707a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:47.259733 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3137605-aa55-4872-81c7-adbfe370707a None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:47.270558 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 466/912] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 12:31:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:47.272614 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3137605-aa55-4872-81c7-adbfe370707a None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:47.280654 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:47.334670 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:47.391368 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca2b2ec-8ed5-4a79-8089-bb11c8320b78 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:47.396506 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3137605-aa55-4872-81c7-adbfe370707a None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:47.435227 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6256eaa6-a6e5-4189-ab11-abe84083e0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:47.435870 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256eaa6-a6e5-4189-ab11-abe84083e0de None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:47.436143 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256eaa6-a6e5-4189-ab11-abe84083e0de None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:47.436410 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256eaa6-a6e5-4189-ab11-abe84083e0de None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:47.437045 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6256eaa6-a6e5-4189-ab11-abe84083e0de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:47.437340 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6256eaa6-a6e5-4189-ab11-abe84083e0de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:47.438699 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6256eaa6-a6e5-4189-ab11-abe84083e0de None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:47.444261 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256bfd3c-36fd-407f-ba83-709e632d4feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:47.445166 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256bfd3c-36fd-407f-ba83-709e632d4feb None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:47.445659 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256bfd3c-36fd-407f-ba83-709e632d4feb None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:47.446183 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256bfd3c-36fd-407f-ba83-709e632d4feb None admin] PATH_INFO: `/v3/services` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:47.447018 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256bfd3c-36fd-407f-ba83-709e632d4feb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:47.447617 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256bfd3c-36fd-407f-ba83-709e632d4feb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:47.449254 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256bfd3c-36fd-407f-ba83-709e632d4feb None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:47.451277 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 447/913] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:31:47.454972 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 467/914] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 12:31:47 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:47.456662 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6256eaa6-a6e5-4189-ab11-abe84083e0de None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:47.462256 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256bfd3c-36fd-407f-ba83-709e632d4feb None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:47.546523 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:47.564536 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6256eaa6-a6e5-4189-ab11-abe84083e0de None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:47.586608 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256bfd3c-36fd-407f-ba83-709e632d4feb None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:47.603551 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:47.612981 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b336a7-06f3-4d22-9db5-befe380c95ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:47.614107 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b336a7-06f3-4d22-9db5-befe380c95ed None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:47.614829 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b336a7-06f3-4d22-9db5-befe380c95ed None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:47.615798 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b336a7-06f3-4d22-9db5-befe380c95ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:47.625575 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b336a7-06f3-4d22-9db5-befe380c95ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:47.627364 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b336a7-06f3-4d22-9db5-befe380c95ed None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:47.627697 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e42038-2600-4b6a-b80c-e2d0dfd3b503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:47.628603 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e42038-2600-4b6a-b80c-e2d0dfd3b503 None admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:47.628957 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e42038-2600-4b6a-b80c-e2d0dfd3b503 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:47.629305 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e42038-2600-4b6a-b80c-e2d0dfd3b503 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:47.630053 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e42038-2600-4b6a-b80c-e2d0dfd3b503 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:47.632473 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e42038-2600-4b6a-b80c-e2d0dfd3b503 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:47.642888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-d8b336a7-06f3-4d22-9db5-befe380c95ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:47.642888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:47.642888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:47.642888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:47.642888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:47.642888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:47.642888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:47.642888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:47.642888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:47.642888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:47.642888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:47.642888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:47.642888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 12:31:47.642888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 12:31:47.642888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 12:31:47.644002 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 12:31:47.644002 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:47.644002 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:47.644002 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:47.644002 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:47.644002 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:47.644002 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:47.644002 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:47.644002 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:47.644002 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:47.644002 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:47.644002 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:47.644002 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:47.644002 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:47.644002 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:47.644002 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:47.644002 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:47.644885 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 12:31:47.644885 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 12:31:47.644885 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 12:31:47.644885 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 12:31:47.644885 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 12:31:47.644885 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 12:31:47.644885 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:47.644885 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:47.648170 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 448/915] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:31:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:31:47.653809 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b336a7-06f3-4d22-9db5-befe380c95ed None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:47.714984 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 468/916] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:31:47 2022] POST /identity/v3/endpoints => generated 346 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:31:47.782067 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b336a7-06f3-4d22-9db5-befe380c95ed None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:47.836418 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fdc1a4-68f6-4f2d-adcb-0f0cc4f928d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:47.837439 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fdc1a4-68f6-4f2d-adcb-0f0cc4f928d6 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:47.837924 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fdc1a4-68f6-4f2d-adcb-0f0cc4f928d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:47.838399 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fdc1a4-68f6-4f2d-adcb-0f0cc4f928d6 None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:47.839299 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fdc1a4-68f6-4f2d-adcb-0f0cc4f928d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:47.839904 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fdc1a4-68f6-4f2d-adcb-0f0cc4f928d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:47.841579 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fdc1a4-68f6-4f2d-adcb-0f0cc4f928d6 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:47.853445 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:47.859041 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 449/917] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 12:31:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:31:47.868489 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e42038-2600-4b6a-b80c-e2d0dfd3b503 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:47.920782 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a7d31-5e26-4dce-9c82-4ba6295c4172 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:47.920970 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a7d31-5e26-4dce-9c82-4ba6295c4172 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:47.921248 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a7d31-5e26-4dce-9c82-4ba6295c4172 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:47.922583 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 450/918] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 12:31:47.929139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9e471-b3c4-4bf4-a4f6-a8f79753c3bc None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:47.929416 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9e471-b3c4-4bf4-a4f6-a8f79753c3bc None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:47.929690 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9e471-b3c4-4bf4-a4f6-a8f79753c3bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:47.950605 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-19d9e471-b3c4-4bf4-a4f6-a8f79753c3bc None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:47.951978 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-19d9e471-b3c4-4bf4-a4f6-a8f79753c3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:47.991800 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e42038-2600-4b6a-b80c-e2d0dfd3b503 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:48.028288 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa56354-70b8-4a3b-a1b4-c2fff57ccf10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:48.029912 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa56354-70b8-4a3b-a1b4-c2fff57ccf10 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:48.030405 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa56354-70b8-4a3b-a1b4-c2fff57ccf10 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:48.030850 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa56354-70b8-4a3b-a1b4-c2fff57ccf10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:48.031789 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa56354-70b8-4a3b-a1b4-c2fff57ccf10 None admin] RBAC: Adding query filter params (role.id=a7c026bed2224838b2a71641eb61b9d0, scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7, user.id=2e6e4387a7c24bbc82c22eefe2a3d138) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:48.032332 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa56354-70b8-4a3b-a1b4-c2fff57ccf10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:48.034048 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa56354-70b8-4a3b-a1b4-c2fff57ccf10 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:48.038350 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 451/919] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 12:31:48.045306 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 469/920] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:31:47 2022] GET /identity/v3/role_assignments?user.id=2e6e4387a7c24bbc82c22eefe2a3d138&scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7&role.id=a7c026bed2224838b2a71641eb61b9d0 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:48.309640 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d9e471-b3c4-4bf4-a4f6-a8f79753c3bc None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:48.411252 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d9e471-b3c4-4bf4-a4f6-a8f79753c3bc None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:48.440256 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:48.446055 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a450dd-d86e-46fe-aa3c-2604881fab65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:48.446698 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a450dd-d86e-46fe-aa3c-2604881fab65 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:48.447226 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a450dd-d86e-46fe-aa3c-2604881fab65 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:48.447532 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a450dd-d86e-46fe-aa3c-2604881fab65 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:48.448108 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a450dd-d86e-46fe-aa3c-2604881fab65 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:48.449466 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a450dd-d86e-46fe-aa3c-2604881fab65 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:48.458341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-f2a450dd-d86e-46fe-aa3c-2604881fab65 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 12:31:48.458341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:48.458341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 12:31:48.458341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 12:31:48.458341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 12:31:48.458341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 12:31:48.458341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 12:31:48.458341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:31:48.458341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 12:31:48.458341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:48.458341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 12:31:48.458341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:48.458341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:48.458341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:48.458341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:48.458341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:48.458341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:48.462220 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:48.462220 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:48.462220 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:48.462220 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:48.462220 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:48.462220 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:48.462220 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 12:31:48.462220 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 12:31:48.462220 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 12:31:48.462220 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 12:31:48.462220 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:48.462220 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:48.462220 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:48.462220 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:48.462220 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:48.462220 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:48.463031 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:48.463031 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:48.463031 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:48.463031 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:48.463031 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:48.463031 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:48.463031 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:48.463031 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:48.463031 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:48.463031 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:48.463031 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 12:31:48.463031 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:31:48.463031 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 12:31:48.463031 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:48.463031 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 452/921] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Oct 3 12:31:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:48.466211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa56354-70b8-4a3b-a1b4-c2fff57ccf10 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:48.503712 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:48.573699 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa56354-70b8-4a3b-a1b4-c2fff57ccf10 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:48.609998 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4674df4-8cd2-43e6-9fe3-b563b23648bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:48.610608 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4674df4-8cd2-43e6-9fe3-b563b23648bb None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:48.610882 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4674df4-8cd2-43e6-9fe3-b563b23648bb None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:48.611148 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4674df4-8cd2-43e6-9fe3-b563b23648bb None admin] PATH_INFO: `/v3/services` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:48.611827 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4674df4-8cd2-43e6-9fe3-b563b23648bb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:48.612127 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4674df4-8cd2-43e6-9fe3-b563b23648bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:48.613467 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4674df4-8cd2-43e6-9fe3-b563b23648bb None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:48.619459 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 470/922] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Oct 3 12:31:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:48.624858 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a450dd-d86e-46fe-aa3c-2604881fab65 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:48.728529 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a450dd-d86e-46fe-aa3c-2604881fab65 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:48.767027 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ffaa62-da44-40f1-aadf-a74aaae037d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:48.768097 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ffaa62-da44-40f1-aadf-a74aaae037d8 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:48.768582 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ffaa62-da44-40f1-aadf-a74aaae037d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:48.769060 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ffaa62-da44-40f1-aadf-a74aaae037d8 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:48.769905 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ffaa62-da44-40f1-aadf-a74aaae037d8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:48.770424 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ffaa62-da44-40f1-aadf-a74aaae037d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:48.772075 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ffaa62-da44-40f1-aadf-a74aaae037d8 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:48.777800 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 453/923] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Oct 3 12:31:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:48.787385 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4674df4-8cd2-43e6-9fe3-b563b23648bb None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:48.883805 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:48.887550 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4674df4-8cd2-43e6-9fe3-b563b23648bb None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:48.921711 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c05a358-d6bc-4504-a8cf-fa023aa7aaf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:48.922317 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05a358-d6bc-4504-a8cf-fa023aa7aaf4 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:48.922594 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05a358-d6bc-4504-a8cf-fa023aa7aaf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:48.922863 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05a358-d6bc-4504-a8cf-fa023aa7aaf4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:48.923584 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c05a358-d6bc-4504-a8cf-fa023aa7aaf4 None admin] RBAC: Adding query filter params (interface=public, service_id=570bb19fd12e48859e3967f3eaa7a0dc, region_id=RegionOne) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:48.923888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c05a358-d6bc-4504-a8cf-fa023aa7aaf4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:48.925298 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c05a358-d6bc-4504-a8cf-fa023aa7aaf4 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:48.931594 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 471/924] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Oct 3 12:31:48 2022] GET /identity/v3/endpoints?service_id=570bb19fd12e48859e3967f3eaa7a0dc&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:48.937108 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ffaa62-da44-40f1-aadf-a74aaae037d8 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:49.004134 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:49.050473 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ffaa62-da44-40f1-aadf-a74aaae037d8 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:49.085554 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c564bcf3-baf5-45da-bab8-8f5ef2d84175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:49.086170 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564bcf3-baf5-45da-bab8-8f5ef2d84175 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:49.086445 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564bcf3-baf5-45da-bab8-8f5ef2d84175 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:49.086712 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564bcf3-baf5-45da-bab8-8f5ef2d84175 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:49.087333 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c564bcf3-baf5-45da-bab8-8f5ef2d84175 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:49.088690 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c564bcf3-baf5-45da-bab8-8f5ef2d84175 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:49.095266 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 454/925] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Oct 3 12:31:48 2022] GET /identity/v3/services => generated 1497 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:31:49.112129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b253c226-3c45-4f9b-ba3f-a51c081bf227 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:49.113117 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b253c226-3c45-4f9b-ba3f-a51c081bf227 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:49.113617 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b253c226-3c45-4f9b-ba3f-a51c081bf227 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:49.116239 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 472/926] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:49.122016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec50b67-93e8-47ac-a5eb-20a64f2072cd None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:49.122333 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec50b67-93e8-47ac-a5eb-20a64f2072cd None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:49.122569 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec50b67-93e8-47ac-a5eb-20a64f2072cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:49.143476 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-fec50b67-93e8-47ac-a5eb-20a64f2072cd None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:49.144689 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-fec50b67-93e8-47ac-a5eb-20a64f2072cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:49.224487 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 455/927] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:31:49.507366 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b253c226-3c45-4f9b-ba3f-a51c081bf227 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:49.615194 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b253c226-3c45-4f9b-ba3f-a51c081bf227 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:49.650263 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295d9b19-7f01-4916-a0a9-b099bd041832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:49.650889 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d9b19-7f01-4916-a0a9-b099bd041832 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:49.651164 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d9b19-7f01-4916-a0a9-b099bd041832 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:49.651464 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d9b19-7f01-4916-a0a9-b099bd041832 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:49.652038 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295d9b19-7f01-4916-a0a9-b099bd041832 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:49.653379 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295d9b19-7f01-4916-a0a9-b099bd041832 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:49.662830 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-295d9b19-7f01-4916-a0a9-b099bd041832 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 03 12:31:49.662830 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:49.662830 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 12:31:49.662830 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 12:31:49.662830 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 12:31:49.662830 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 12:31:49.662830 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 12:31:49.662830 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:31:49.662830 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 03 12:31:49.662830 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:49.662830 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 12:31:49.662830 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:49.662830 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:49.662830 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:49.662830 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:49.662830 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:49.663843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:49.663843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:49.663843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:49.663843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:49.663843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:49.663843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:49.663843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:49.663843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 12:31:49.663843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 12:31:49.663843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 12:31:49.663843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 12:31:49.663843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:49.663843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:49.663843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:49.663843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:49.663843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:49.663843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:49.664736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:49.664736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:49.664736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:49.664736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:49.664736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:49.664736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:49.664736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:49.664736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:49.664736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:49.664736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:49.664736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 12:31:49.664736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:31:49.664736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 03 12:31:49.664736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:49.664736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 473/928] 10.222.0.33 () {56 vars in 1132 bytes} [Mon Oct 3 12:31:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:49.672180 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec50b67-93e8-47ac-a5eb-20a64f2072cd None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:49.730608 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:49.779665 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec50b67-93e8-47ac-a5eb-20a64f2072cd None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:49.820607 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c465efae-f164-4a25-95cc-213f606638bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:49.821232 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465efae-f164-4a25-95cc-213f606638bb None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:49.821567 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465efae-f164-4a25-95cc-213f606638bb None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:49.821845 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465efae-f164-4a25-95cc-213f606638bb None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:49.822471 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c465efae-f164-4a25-95cc-213f606638bb None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:49.822774 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c465efae-f164-4a25-95cc-213f606638bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:49.824178 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c465efae-f164-4a25-95cc-213f606638bb None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:49.829323 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 456/929] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Oct 3 12:31:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:49.835395 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295d9b19-7f01-4916-a0a9-b099bd041832 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:49.868898 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:49.943982 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295d9b19-7f01-4916-a0a9-b099bd041832 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:49.982016 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc589b-5424-4032-88ae-0fc769ea2cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:49.982670 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc589b-5424-4032-88ae-0fc769ea2cbc None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:49.982948 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc589b-5424-4032-88ae-0fc769ea2cbc None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:49.983265 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc589b-5424-4032-88ae-0fc769ea2cbc None admin] PATH_INFO: `/v3/services` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:49.983926 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc589b-5424-4032-88ae-0fc769ea2cbc None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:49.984223 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc589b-5424-4032-88ae-0fc769ea2cbc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:49.985572 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc589b-5424-4032-88ae-0fc769ea2cbc None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:49.991687 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 474/930] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Oct 3 12:31:49 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:31:50.000776 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c465efae-f164-4a25-95cc-213f606638bb None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:50.113937 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c465efae-f164-4a25-95cc-213f606638bb None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:50.131948 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:50.149433 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29022e47-4f18-4d60-baf7-9c6b0f4632bf None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:50.149953 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29022e47-4f18-4d60-baf7-9c6b0f4632bf None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:50.149953 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29022e47-4f18-4d60-baf7-9c6b0f4632bf None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:50.153574 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 475/931] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 12:31:50.153670 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c9b97-c556-4c67-9a56-de8cbcb79b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:50.153933 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c9b97-c556-4c67-9a56-de8cbcb79b54 None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:50.154363 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c9b97-c556-4c67-9a56-de8cbcb79b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:50.154785 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c9b97-c556-4c67-9a56-de8cbcb79b54 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:50.155631 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9c9b97-c556-4c67-9a56-de8cbcb79b54 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:50.157195 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9c9b97-c556-4c67-9a56-de8cbcb79b54 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:50.161868 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff3a488-9c4a-45f3-99aa-20b15d1dc52e None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:50.162324 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff3a488-9c4a-45f3-99aa-20b15d1dc52e None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:50.162678 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff3a488-9c4a-45f3-99aa-20b15d1dc52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:50.181868 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-fff3a488-9c4a-45f3-99aa-20b15d1dc52e None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:50.183042 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-fff3a488-9c4a-45f3-99aa-20b15d1dc52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:50.252197 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 457/932] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:31:49 2022] POST /identity/v3/endpoints => generated 347 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:31:50.285028 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 476/933] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:31:50.375760 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6f73f-b470-4e07-ac08-bd8cb9cf5768 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:50.376063 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6f73f-b470-4e07-ac08-bd8cb9cf5768 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:50.376296 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6f73f-b470-4e07-ac08-bd8cb9cf5768 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:50.377974 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 458/934] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:50.384527 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900f72a-0f2c-4342-b1fb-eca2acc1c811 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:50.384863 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900f72a-0f2c-4342-b1fb-eca2acc1c811 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:50.385162 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900f72a-0f2c-4342-b1fb-eca2acc1c811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:50.405326 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-c900f72a-0f2c-4342-b1fb-eca2acc1c811 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:50.406547 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-c900f72a-0f2c-4342-b1fb-eca2acc1c811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:50.420212 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:50.479354 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 477/935] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:31:50.560850 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6f73f-b470-4e07-ac08-bd8cb9cf5768 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:50.674424 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6f73f-b470-4e07-ac08-bd8cb9cf5768 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:50.712701 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd2e98d-1fe3-4552-a9ce-d56a3a196fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:50.713648 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2e98d-1fe3-4552-a9ce-d56a3a196fd7 None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:50.714134 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2e98d-1fe3-4552-a9ce-d56a3a196fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:50.714633 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2e98d-1fe3-4552-a9ce-d56a3a196fd7 None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:50.715500 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd2e98d-1fe3-4552-a9ce-d56a3a196fd7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:50.717140 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd2e98d-1fe3-4552-a9ce-d56a3a196fd7 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:50.720345 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: WARNING keystone.server.flask.common [None req-abd2e98d-1fe3-4552-a9ce-d56a3a196fd7 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 03 12:31:50.764269 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c900f72a-0f2c-4342-b1fb-eca2acc1c811 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:50.783614 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.sql.core [None req-abd2e98d-1fe3-4552-a9ce-d56a3a196fd7 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 03 12:31:50.783614 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 03 12:31:50.783614 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [parameters: {'id': '64e237f8c4e342dc9fecc01fa18362b0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 03 12:31:50.783614 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81805) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 03 12:31:50.807670 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-abd2e98d-1fe3-4552-a9ce-d56a3a196fd7 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 64e237f8c4e342dc9fecc01fa18362b0.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 64e237f8c4e342dc9fecc01fa18362b0. Oct 03 12:31:50.807670 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:50.807670 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 03 12:31:50.807670 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 03 12:31:50.807670 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 03 12:31:50.807670 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 03 12:31:50.807670 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 03 12:31:50.807670 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application result = self._query(query) Oct 03 12:31:50.807670 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 03 12:31:50.807670 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application conn.query(q) Oct 03 12:31:50.807670 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 03 12:31:50.807670 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 03 12:31:50.807670 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 03 12:31:50.808534 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application result.read() Oct 03 12:31:50.808534 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 03 12:31:50.808534 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 03 12:31:50.808534 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 03 12:31:50.808534 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 03 12:31:50.808534 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 03 12:31:50.808534 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 03 12:31:50.808534 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 03 12:31:50.808534 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 03 12:31:50.808534 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 03 12:31:50.808534 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:50.808534 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 03 12:31:50.808534 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:50.808534 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:50.808534 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 03 12:31:50.808534 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 03 12:31:50.808534 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 03 12:31:50.808534 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 03 12:31:50.809493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 03 12:31:50.809493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application next(self.gen) Oct 03 12:31:50.809493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 03 12:31:50.809493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application yield resource Oct 03 12:31:50.809493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 03 12:31:50.809493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application next(self.gen) Oct 03 12:31:50.809493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 03 12:31:50.809493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self.session.rollback() Oct 03 12:31:50.809493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 03 12:31:50.809493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self.force_reraise() Oct 03 12:31:50.809493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 03 12:31:50.809493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise self.value Oct 03 12:31:50.809493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 03 12:31:50.809493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 03 12:31:50.809493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 03 12:31:50.809493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application session.commit() Oct 03 12:31:50.809493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 03 12:31:50.810425 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 03 12:31:50.810425 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 03 12:31:50.810425 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._prepare_impl() Oct 03 12:31:50.810425 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 03 12:31:50.810425 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self.session.flush() Oct 03 12:31:50.810425 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 03 12:31:50.810425 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._flush(objects) Oct 03 12:31:50.810425 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 03 12:31:50.810425 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 03 12:31:50.810425 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 03 12:31:50.810425 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application compat.raise_( Oct 03 12:31:50.810425 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 03 12:31:50.810425 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception Oct 03 12:31:50.810425 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 03 12:31:50.810425 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application flush_context.execute() Oct 03 12:31:50.810425 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 03 12:31:50.810425 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rec.execute(self) Oct 03 12:31:50.811335 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 03 12:31:50.811335 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 03 12:31:50.811335 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 03 12:31:50.811335 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 03 12:31:50.811335 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 03 12:31:50.811335 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 03 12:31:50.811335 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 03 12:31:50.811335 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 03 12:31:50.811335 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 03 12:31:50.811335 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 03 12:31:50.811335 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 03 12:31:50.811335 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 03 12:31:50.811335 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 03 12:31:50.811335 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 03 12:31:50.811335 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 03 12:31:50.812152 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 03 12:31:50.812152 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 03 12:31:50.812152 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception Oct 03 12:31:50.812152 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 03 12:31:50.812152 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 03 12:31:50.812152 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 03 12:31:50.812152 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 03 12:31:50.812152 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 03 12:31:50.812152 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application result = self._query(query) Oct 03 12:31:50.812152 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 03 12:31:50.812152 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application conn.query(q) Oct 03 12:31:50.812152 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 03 12:31:50.812152 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 03 12:31:50.812152 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 03 12:31:50.812152 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application result.read() Oct 03 12:31:50.812152 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 03 12:31:50.812152 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 03 12:31:50.814771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 03 12:31:50.814771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 03 12:31:50.814771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 03 12:31:50.814771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 03 12:31:50.814771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 03 12:31:50.814771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 03 12:31:50.814771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 03 12:31:50.814771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 03 12:31:50.814771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [parameters: {'id': '64e237f8c4e342dc9fecc01fa18362b0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 03 12:31:50.814771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 03 12:31:50.814771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:50.814771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 12:31:50.814771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:50.814771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:50.814771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 03 12:31:50.817188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 03 12:31:50.817188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 03 12:31:50.817188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 03 12:31:50.817188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 03 12:31:50.817188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:50.817188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 12:31:50.817188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:50.817188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:50.817188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:50.817188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:50.817188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:50.817188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:50.817188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:50.817188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:50.817188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:50.817188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:50.817188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:50.819513 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:50.819513 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 03 12:31:50.819513 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 03 12:31:50.819513 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:50.819513 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:50.819513 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 03 12:31:50.819513 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 03 12:31:50.819513 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 64e237f8c4e342dc9fecc01fa18362b0. Oct 03 12:31:50.819513 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:50.819513 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 459/936] 10.222.0.33 () {60 vars in 1140 bytes} [Mon Oct 3 12:31:50 2022] POST /identity/v3/projects => generated 262 bytes in 253 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:50.819513 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd2e98d-1fe3-4552-a9ce-d56a3a196fd7 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:50.861509 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:50.885316 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c900f72a-0f2c-4342-b1fb-eca2acc1c811 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:50.926528 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61e8eaa-3668-4f07-ac1e-3a6e5c679420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:50.927237 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e8eaa-3668-4f07-ac1e-3a6e5c679420 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:50.927593 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e8eaa-3668-4f07-ac1e-3a6e5c679420 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:50.927870 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e8eaa-3668-4f07-ac1e-3a6e5c679420 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:50.940894 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61e8eaa-3668-4f07-ac1e-3a6e5c679420 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:31:50.947468 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-f61e8eaa-3668-4f07-ac1e-3a6e5c679420 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 12:31:50.947468 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:50.947468 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:50.947468 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:50.947468 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:50.947468 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:50.947468 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:50.947468 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:50.947468 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:50.947468 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:50.947468 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:50.947468 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:50.947468 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:31:50.947468 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:31:50.947468 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:31:50.947468 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:31:50.948411 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:31:50.948411 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:31:50.948411 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:31:50.948411 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:31:50.948411 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:50.948411 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:50.948411 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:50.948411 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:50.948411 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:50.948411 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:50.948411 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:50.948411 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:50.948411 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:50.948411 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:50.948411 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:50.948411 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:50.949309 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:50.949309 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:50.949309 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:50.949309 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:50.949309 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:31:50.949309 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:31:50.949309 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:31:50.949309 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:31:50.949309 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:31:50.949309 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:31:50.949309 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 12:31:50.949309 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:50.949309 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 478/937] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Oct 3 12:31:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:50.955898 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61e8eaa-3668-4f07-ac1e-3a6e5c679420 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:50.969456 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:50.970773 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd2e98d-1fe3-4552-a9ce-d56a3a196fd7 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:51.009328 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c735c-dc3b-48b7-a622-dd6fb487ef2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:51.009968 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c735c-dc3b-48b7-a622-dd6fb487ef2e None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:51.010244 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c735c-dc3b-48b7-a622-dd6fb487ef2e None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:51.010514 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c735c-dc3b-48b7-a622-dd6fb487ef2e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:51.018010 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019c735c-dc3b-48b7-a622-dd6fb487ef2e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:51.019945 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019c735c-dc3b-48b7-a622-dd6fb487ef2e None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:51.032510 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-019c735c-dc3b-48b7-a622-dd6fb487ef2e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:51.032510 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:51.032510 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:51.032510 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:51.032510 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:51.032510 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:51.032510 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:51.032510 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:51.032510 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:51.032510 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:51.032510 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:51.032510 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:51.032510 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 12:31:51.032510 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 12:31:51.032510 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 12:31:51.033447 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 12:31:51.033447 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:51.033447 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:51.033447 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:51.033447 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:51.033447 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:51.033447 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:51.033447 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:51.033447 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:51.033447 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:51.033447 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:51.033447 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:51.033447 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:51.033447 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:51.033447 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:51.033447 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:51.033447 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:51.034327 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 12:31:51.034327 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 12:31:51.034327 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 12:31:51.034327 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 12:31:51.034327 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 12:31:51.034327 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 12:31:51.034327 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:51.034327 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:51.034327 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 460/938] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:31:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:51.040187 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c735c-dc3b-48b7-a622-dd6fb487ef2e None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:51.089154 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61e8eaa-3668-4f07-ac1e-3a6e5c679420 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:51.126860 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcc2e50-87e0-4eb1-b709-09a44f645872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:51.127650 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcc2e50-87e0-4eb1-b709-09a44f645872 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:51.128028 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcc2e50-87e0-4eb1-b709-09a44f645872 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:51.128567 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcc2e50-87e0-4eb1-b709-09a44f645872 None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:51.129168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcc2e50-87e0-4eb1-b709-09a44f645872 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:51.129544 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcc2e50-87e0-4eb1-b709-09a44f645872 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:51.131646 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcc2e50-87e0-4eb1-b709-09a44f645872 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:51.140902 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:51.142405 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 479/939] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Oct 3 12:31:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:51.146044 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcc2e50-87e0-4eb1-b709-09a44f645872 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:51.160319 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c735c-dc3b-48b7-a622-dd6fb487ef2e None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:51.212104 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6203a1-54d9-4644-874c-c8d205314319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:51.212873 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6203a1-54d9-4644-874c-c8d205314319 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:51.213155 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6203a1-54d9-4644-874c-c8d205314319 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:51.213431 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6203a1-54d9-4644-874c-c8d205314319 None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:51.214094 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6203a1-54d9-4644-874c-c8d205314319 None admin] RBAC: Adding query filter params (name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:51.214394 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6203a1-54d9-4644-874c-c8d205314319 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:51.216512 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6203a1-54d9-4644-874c-c8d205314319 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:51.241625 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:51.252979 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 461/940] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Oct 3 12:31:51 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:51.297250 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcc2e50-87e0-4eb1-b709-09a44f645872 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:51.336566 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89e638a-e530-4d2d-9ffc-cd53fdec930c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:51.337290 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e638a-e530-4d2d-9ffc-cd53fdec930c None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:51.337615 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e638a-e530-4d2d-9ffc-cd53fdec930c None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:51.337897 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e638a-e530-4d2d-9ffc-cd53fdec930c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:51.350884 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89e638a-e530-4d2d-9ffc-cd53fdec930c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:51.352374 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89e638a-e530-4d2d-9ffc-cd53fdec930c None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:51.368605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-f89e638a-e530-4d2d-9ffc-cd53fdec930c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:51.368605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:51.368605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:51.368605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:51.368605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:51.368605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:51.368605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:51.368605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:51.368605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:51.368605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:51.368605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:51.368605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:51.368605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:51.368605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:51.368605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:51.368605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:51.369611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:51.369611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:51.369611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:51.369611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:51.369611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:51.369611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:51.369611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:51.369611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:51.369611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:51.369611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:51.369611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:51.369611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:51.369611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:51.369611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:51.369611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:51.369611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:51.369611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:51.370488 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:51.370488 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:51.370488 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:51.370488 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 480/941] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:31:51.377295 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89e638a-e530-4d2d-9ffc-cd53fdec930c None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:51.377475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f2cd8-e6c1-4d42-a2d3-fbb80c9d821e None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:51.377475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f2cd8-e6c1-4d42-a2d3-fbb80c9d821e None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:51.377475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f2cd8-e6c1-4d42-a2d3-fbb80c9d821e None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:51.379099 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 462/942] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:51.385167 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b9651-b92f-4e21-b087-34f07f90bf20 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:51.385485 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b9651-b92f-4e21-b087-34f07f90bf20 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:51.385733 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b9651-b92f-4e21-b087-34f07f90bf20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:51.405981 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-fc6b9651-b92f-4e21-b087-34f07f90bf20 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:51.407592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-fc6b9651-b92f-4e21-b087-34f07f90bf20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:51.498290 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 463/943] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 12:31:51.508312 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89e638a-e530-4d2d-9ffc-cd53fdec930c None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:51.532804 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:51.548284 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea45ad-ebbb-4a1e-97eb-34896dbbad7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:51.550386 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea45ad-ebbb-4a1e-97eb-34896dbbad7c None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:51.550386 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea45ad-ebbb-4a1e-97eb-34896dbbad7c None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:51.550386 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea45ad-ebbb-4a1e-97eb-34896dbbad7c None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:51.550386 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea45ad-ebbb-4a1e-97eb-34896dbbad7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:51.550386 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea45ad-ebbb-4a1e-97eb-34896dbbad7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:51.551831 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea45ad-ebbb-4a1e-97eb-34896dbbad7c None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:51.565506 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 481/944] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:31:51.572646 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b9651-b92f-4e21-b087-34f07f90bf20 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:51.656974 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:51.682979 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b9651-b92f-4e21-b087-34f07f90bf20 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:51.720817 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525feaaa-7839-4193-bf94-4dc3eaa4c32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:51.721806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525feaaa-7839-4193-bf94-4dc3eaa4c32f None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:51.722292 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525feaaa-7839-4193-bf94-4dc3eaa4c32f None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:51.722784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525feaaa-7839-4193-bf94-4dc3eaa4c32f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:51.731216 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525feaaa-7839-4193-bf94-4dc3eaa4c32f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:51.732915 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525feaaa-7839-4193-bf94-4dc3eaa4c32f None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:51.745128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-525feaaa-7839-4193-bf94-4dc3eaa4c32f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 12:31:51.745128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:51.745128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:51.745128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:51.745128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:51.745128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:51.745128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:51.745128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:51.745128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:51.745128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:51.745128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:51.745128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:51.745128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 12:31:51.745128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 12:31:51.745128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 12:31:51.745128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 12:31:51.746070 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:51.746070 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:51.746070 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 12:31:51.746070 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:51.746070 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 12:31:51.746070 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:51.746070 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:51.746070 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:51.746070 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:51.746070 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:51.746070 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:51.746070 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:51.746070 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:51.746070 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:51.746070 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:51.746070 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:51.746070 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:51.746070 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:51.747037 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:51.747037 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:51.747037 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 12:31:51.747037 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 12:31:51.747037 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 12:31:51.747037 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 12:31:51.747037 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 12:31:51.747037 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 12:31:51.747037 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 12:31:51.747037 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:51.749568 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 464/945] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Oct 3 12:31:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:51.754768 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea45ad-ebbb-4a1e-97eb-34896dbbad7c None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:51.806367 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525feaaa-7839-4193-bf94-4dc3eaa4c32f None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:51.860472 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea45ad-ebbb-4a1e-97eb-34896dbbad7c None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:51.900921 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe4619e-bc39-4f4a-913c-1ea7f24e4d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:51.902181 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe4619e-bc39-4f4a-913c-1ea7f24e4d6d None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:51.902711 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe4619e-bc39-4f4a-913c-1ea7f24e4d6d None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:51.903182 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe4619e-bc39-4f4a-913c-1ea7f24e4d6d None admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:51.904134 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe4619e-bc39-4f4a-913c-1ea7f24e4d6d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:51.905040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe4619e-bc39-4f4a-913c-1ea7f24e4d6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:51.906969 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe4619e-bc39-4f4a-913c-1ea7f24e4d6d None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:51.918157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525feaaa-7839-4193-bf94-4dc3eaa4c32f None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:51.931283 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 482/946] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Oct 3 12:31:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:31:51.936615 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe4619e-bc39-4f4a-913c-1ea7f24e4d6d None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:51.963047 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a014ab-ec56-4bc8-a7e6-83d865fb3c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:51.964065 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a014ab-ec56-4bc8-a7e6-83d865fb3c98 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:51.964669 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a014ab-ec56-4bc8-a7e6-83d865fb3c98 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:51.965201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a014ab-ec56-4bc8-a7e6-83d865fb3c98 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:51.966068 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a014ab-ec56-4bc8-a7e6-83d865fb3c98 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:51.967753 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a014ab-ec56-4bc8-a7e6-83d865fb3c98 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:51.978445 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:51.980016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-36a014ab-ec56-4bc8-a7e6-83d865fb3c98 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 12:31:51.980016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:51.980016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 12:31:51.980016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 12:31:51.980016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 12:31:51.980016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 12:31:51.980016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 12:31:51.980016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:31:51.980016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 12:31:51.980016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:51.980016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 12:31:51.980016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:51.980016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:51.980016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:51.980016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:51.980016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:51.980016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:51.980988 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:51.980988 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:51.980988 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:51.980988 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:51.980988 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:51.980988 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:51.980988 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 12:31:51.980988 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 12:31:51.980988 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 12:31:51.980988 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 12:31:51.980988 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:51.980988 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:51.980988 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:51.980988 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:51.980988 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:51.980988 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:51.981888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:51.981888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:51.981888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:51.981888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:51.981888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:51.981888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:51.981888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:51.981888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:51.981888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:51.981888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:51.981888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 12:31:51.981888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:31:51.981888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 12:31:51.981888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:51.984684 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 465/947] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Oct 3 12:31:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:31:51.990406 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a014ab-ec56-4bc8-a7e6-83d865fb3c98 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:51.994202 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:52.050157 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe4619e-bc39-4f4a-913c-1ea7f24e4d6d None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:52.091173 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce0be08-962f-4a8a-95c6-5a02cab9bd6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:52.092503 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0be08-962f-4a8a-95c6-5a02cab9bd6d None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:52.092960 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0be08-962f-4a8a-95c6-5a02cab9bd6d None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:52.093652 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0be08-962f-4a8a-95c6-5a02cab9bd6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:52.107549 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce0be08-962f-4a8a-95c6-5a02cab9bd6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:52.109750 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce0be08-962f-4a8a-95c6-5a02cab9bd6d None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:52.118445 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a014ab-ec56-4bc8-a7e6-83d865fb3c98 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:52.130028 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-5ce0be08-962f-4a8a-95c6-5a02cab9bd6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:52.130028 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:52.130028 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:52.130028 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:52.130028 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:52.130028 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:52.130028 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:52.130028 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:52.130028 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:52.130028 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:52.130028 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:52.130028 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:52.130028 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:52.130028 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:52.130028 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:52.130028 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:52.131448 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:52.131448 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:52.131448 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:52.131448 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:52.131448 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:52.131448 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:52.131448 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:52.131448 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:52.131448 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:52.131448 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:52.131448 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:52.131448 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:52.131448 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:52.131448 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:52.131448 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:52.131448 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:52.131448 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:52.132640 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:52.132640 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:52.132640 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:52.134592 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 483/948] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:52.139984 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce0be08-962f-4a8a-95c6-5a02cab9bd6d None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:52.161593 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a4fab-11ba-404d-b5ed-10a2cdc217f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:52.162461 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a4fab-11ba-404d-b5ed-10a2cdc217f6 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:52.162644 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a4fab-11ba-404d-b5ed-10a2cdc217f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:52.163352 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a4fab-11ba-404d-b5ed-10a2cdc217f6 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:52.163668 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a4fab-11ba-404d-b5ed-10a2cdc217f6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:52.163965 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a4fab-11ba-404d-b5ed-10a2cdc217f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:52.165372 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a4fab-11ba-404d-b5ed-10a2cdc217f6 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:52.176645 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 466/949] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Oct 3 12:31:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:52.178145 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a4fab-11ba-404d-b5ed-10a2cdc217f6 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:52.264318 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:52.269046 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:52.275676 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce0be08-962f-4a8a-95c6-5a02cab9bd6d None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:52.313243 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10bc624-b688-4445-9ac2-8a7d8e1c7557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:52.313843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10bc624-b688-4445-9ac2-8a7d8e1c7557 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:52.314123 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10bc624-b688-4445-9ac2-8a7d8e1c7557 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:52.314594 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10bc624-b688-4445-9ac2-8a7d8e1c7557 None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:52.315016 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10bc624-b688-4445-9ac2-8a7d8e1c7557 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:52.315558 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10bc624-b688-4445-9ac2-8a7d8e1c7557 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:52.316865 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10bc624-b688-4445-9ac2-8a7d8e1c7557 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:52.317954 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a4fab-11ba-404d-b5ed-10a2cdc217f6 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:52.333664 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 484/950] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:52.335745 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10bc624-b688-4445-9ac2-8a7d8e1c7557 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:52.369288 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf795a2-b5a0-4bcf-a85f-619287c78de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:52.370225 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf795a2-b5a0-4bcf-a85f-619287c78de0 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:52.370617 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf795a2-b5a0-4bcf-a85f-619287c78de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:52.370958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf795a2-b5a0-4bcf-a85f-619287c78de0 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:52.372117 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf795a2-b5a0-4bcf-a85f-619287c78de0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:52.372546 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf795a2-b5a0-4bcf-a85f-619287c78de0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:52.375050 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf795a2-b5a0-4bcf-a85f-619287c78de0 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:52.381483 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 467/951] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Oct 3 12:31:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:52.388913 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf795a2-b5a0-4bcf-a85f-619287c78de0 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:52.458240 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10bc624-b688-4445-9ac2-8a7d8e1c7557 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:52.495236 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090621bc-04d6-4e43-bbfd-ae9f9ddf8c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:52.496204 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090621bc-04d6-4e43-bbfd-ae9f9ddf8c9d None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:52.496712 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090621bc-04d6-4e43-bbfd-ae9f9ddf8c9d None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:52.497249 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090621bc-04d6-4e43-bbfd-ae9f9ddf8c9d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:52.504395 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090621bc-04d6-4e43-bbfd-ae9f9ddf8c9d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:52.506646 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090621bc-04d6-4e43-bbfd-ae9f9ddf8c9d None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:52.518690 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-090621bc-04d6-4e43-bbfd-ae9f9ddf8c9d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:52.518690 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:52.518690 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:52.518690 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:52.518690 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:52.518690 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:52.518690 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:52.518690 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:52.518690 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:52.518690 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:52.518690 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:52.518690 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:52.518690 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 12:31:52.518690 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 12:31:52.518690 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 12:31:52.519699 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 12:31:52.519699 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:52.519699 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:52.519699 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:52.519699 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:52.519699 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:52.519699 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:52.519699 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:52.519699 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:52.519699 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:52.519699 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:52.519699 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:52.519699 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:52.519699 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:52.519699 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:52.519699 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:52.519699 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:52.520638 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 12:31:52.520638 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 12:31:52.520638 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 12:31:52.520638 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 12:31:52.520638 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 12:31:52.520638 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 12:31:52.520638 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:52.520638 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:52.523783 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 485/952] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:31:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:52.528954 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090621bc-04d6-4e43-bbfd-ae9f9ddf8c9d None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:52.539727 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf795a2-b5a0-4bcf-a85f-619287c78de0 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:52.542498 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:52.571261 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:52.577324 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c39cb6-9d86-4983-9950-61a2602279dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:52.577937 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c39cb6-9d86-4983-9950-61a2602279dd None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:52.578211 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c39cb6-9d86-4983-9950-61a2602279dd None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:52.578481 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c39cb6-9d86-4983-9950-61a2602279dd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:52.579137 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c39cb6-9d86-4983-9950-61a2602279dd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:52.581416 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c39cb6-9d86-4983-9950-61a2602279dd None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:52.632584 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090621bc-04d6-4e43-bbfd-ae9f9ddf8c9d None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:52.665427 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835bed2-0b55-48a6-89d1-240e3e691ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:52.666024 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835bed2-0b55-48a6-89d1-240e3e691ac6 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:52.666302 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835bed2-0b55-48a6-89d1-240e3e691ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:52.666570 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835bed2-0b55-48a6-89d1-240e3e691ac6 None admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:52.667243 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3835bed2-0b55-48a6-89d1-240e3e691ac6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:52.667578 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3835bed2-0b55-48a6-89d1-240e3e691ac6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:52.669053 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3835bed2-0b55-48a6-89d1-240e3e691ac6 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:52.675698 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 468/953] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:31:52 2022] POST /identity/v3/endpoints => generated 349 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:31:52.677369 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7a040-7bac-4106-901e-26a4e07c5fe1 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:52.678284 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7a040-7bac-4106-901e-26a4e07c5fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:52.678685 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7a040-7bac-4106-901e-26a4e07c5fe1 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:52.680225 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 469/954] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:52.684544 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 486/955] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 12:31:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:31:52.689527 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcdc66-b5f3-43b4-a79b-fcc773e86f5f None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:52.689819 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcdc66-b5f3-43b4-a79b-fcc773e86f5f None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:52.690121 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcdc66-b5f3-43b4-a79b-fcc773e86f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:52.698985 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835bed2-0b55-48a6-89d1-240e3e691ac6 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:52.714209 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-bdfcdc66-b5f3-43b4-a79b-fcc773e86f5f None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:52.716213 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-bdfcdc66-b5f3-43b4-a79b-fcc773e86f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:52.793043 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:52.830764 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835bed2-0b55-48a6-89d1-240e3e691ac6 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:52.840384 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 470/956] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:31:52.873986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22254b-f38e-42ee-a679-692d66c35d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:52.874305 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22254b-f38e-42ee-a679-692d66c35d34 None admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:52.874554 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22254b-f38e-42ee-a679-692d66c35d34 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:52.874832 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22254b-f38e-42ee-a679-692d66c35d34 None admin] PATH_INFO: `/v3/projects/4ea2fa8ad01240f7859182ebbe3051c7/users/2e6e4387a7c24bbc82c22eefe2a3d138/roles/a7c026bed2224838b2a71641eb61b9d0` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:52.877209 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b22254b-f38e-42ee-a679-692d66c35d34 None admin] RBAC: Authorizing `identity:create_grant(project_id=4ea2fa8ad01240f7859182ebbe3051c7, user_id=2e6e4387a7c24bbc82c22eefe2a3d138, role_id=a7c026bed2224838b2a71641eb61b9d0)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:52.878630 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b22254b-f38e-42ee-a679-692d66c35d34 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:52.962203 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 487/957] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 12:31:52 2022] PUT /identity/v3/projects/4ea2fa8ad01240f7859182ebbe3051c7/users/2e6e4387a7c24bbc82c22eefe2a3d138/roles/a7c026bed2224838b2a71641eb61b9d0 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:31:53.129793 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfcdc66-b5f3-43b4-a79b-fcc773e86f5f None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:53.250247 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfcdc66-b5f3-43b4-a79b-fcc773e86f5f None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:53.268904 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:53.292887 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a1314-ed10-4800-8ba8-7ec798dc2c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:53.293535 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a1314-ed10-4800-8ba8-7ec798dc2c73 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:53.293816 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a1314-ed10-4800-8ba8-7ec798dc2c73 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:53.294096 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a1314-ed10-4800-8ba8-7ec798dc2c73 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:53.294673 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a1314-ed10-4800-8ba8-7ec798dc2c73 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:53.296139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a1314-ed10-4800-8ba8-7ec798dc2c73 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:53.305855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-1e5a1314-ed10-4800-8ba8-7ec798dc2c73 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 03 12:31:53.305855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:53.305855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 12:31:53.305855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 12:31:53.305855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 12:31:53.305855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 12:31:53.305855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 12:31:53.305855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:31:53.305855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 03 12:31:53.305855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:53.305855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 12:31:53.305855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:53.305855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:53.305855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:53.305855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:53.305855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:53.305855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:53.307150 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:53.307150 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:53.307150 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:53.307150 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:53.307150 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:53.307150 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:53.307150 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 12:31:53.307150 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 12:31:53.307150 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 12:31:53.307150 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 12:31:53.307150 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:53.307150 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:53.307150 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:53.307150 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:53.307150 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:53.307150 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:53.308216 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:53.308216 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:53.308216 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:53.308216 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:53.308216 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:53.308216 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:53.308216 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:53.308216 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:53.308216 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:53.308216 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:53.308216 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 12:31:53.308216 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:31:53.308216 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 03 12:31:53.308216 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:53.310950 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 471/958] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:31:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:53.316743 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22254b-f38e-42ee-a679-692d66c35d34 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:53.423670 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22254b-f38e-42ee-a679-692d66c35d34 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:53.467707 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:53.472431 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb13863-82ca-4230-ac33-07d7b7cfc4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:53.473055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb13863-82ca-4230-ac33-07d7b7cfc4a3 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:53.473331 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb13863-82ca-4230-ac33-07d7b7cfc4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:53.473604 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb13863-82ca-4230-ac33-07d7b7cfc4a3 None admin] PATH_INFO: `/v3/services` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:53.474234 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb13863-82ca-4230-ac33-07d7b7cfc4a3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:53.475195 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb13863-82ca-4230-ac33-07d7b7cfc4a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:53.476538 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb13863-82ca-4230-ac33-07d7b7cfc4a3 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:53.481988 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 488/959] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Oct 3 12:31:53 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:53.487611 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a1314-ed10-4800-8ba8-7ec798dc2c73 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:53.563627 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c663e69-53b1-4786-a2e6-edcdfd110e4f None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:53.563958 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c663e69-53b1-4786-a2e6-edcdfd110e4f None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:53.564176 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c663e69-53b1-4786-a2e6-edcdfd110e4f None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:53.566513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 489/960] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 12:31:53.573397 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036a184-9b6a-4293-8cd8-e356669c3542 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:53.573686 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036a184-9b6a-4293-8cd8-e356669c3542 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:53.573965 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036a184-9b6a-4293-8cd8-e356669c3542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:53.592870 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-c036a184-9b6a-4293-8cd8-e356669c3542 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:53.594204 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-c036a184-9b6a-4293-8cd8-e356669c3542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:53.614602 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a1314-ed10-4800-8ba8-7ec798dc2c73 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:53.654352 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501fd0b-811f-43ba-b90e-8f38450e02ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:53.654965 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501fd0b-811f-43ba-b90e-8f38450e02ff None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:53.655244 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501fd0b-811f-43ba-b90e-8f38450e02ff None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:53.655558 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501fd0b-811f-43ba-b90e-8f38450e02ff None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:53.656185 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501fd0b-811f-43ba-b90e-8f38450e02ff None admin] RBAC: Adding query filter params (type=compute) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:53.656487 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501fd0b-811f-43ba-b90e-8f38450e02ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:53.657859 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501fd0b-811f-43ba-b90e-8f38450e02ff None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:53.663608 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 472/961] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Oct 3 12:31:53 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:53.688779 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 490/962] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 12:31:53.969606 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501fd0b-811f-43ba-b90e-8f38450e02ff None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:54.017484 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:54.081817 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501fd0b-811f-43ba-b90e-8f38450e02ff None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:54.117764 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2706a-e425-4537-aaf1-bf33160cc806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:54.118654 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2706a-e425-4537-aaf1-bf33160cc806 None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:54.119075 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2706a-e425-4537-aaf1-bf33160cc806 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:54.119509 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2706a-e425-4537-aaf1-bf33160cc806 None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:54.120295 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc2706a-e425-4537-aaf1-bf33160cc806 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:54.121851 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc2706a-e425-4537-aaf1-bf33160cc806 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:54.164008 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.sql.core [None req-9bc2706a-e425-4537-aaf1-bf33160cc806 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 03 12:31:54.164008 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 03 12:31:54.164008 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [parameters: {'user_id': 'ee6c164e689d404481bef790683357fb', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 03 12:31:54.164008 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81805) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 03 12:31:54.171947 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-9bc2706a-e425-4537-aaf1-bf33160cc806 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 03 12:31:54.171947 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:54.171947 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 03 12:31:54.171947 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 03 12:31:54.171947 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 03 12:31:54.171947 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 03 12:31:54.171947 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 03 12:31:54.171947 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application result = self._query(query) Oct 03 12:31:54.171947 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 03 12:31:54.171947 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application conn.query(q) Oct 03 12:31:54.171947 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 03 12:31:54.171947 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 03 12:31:54.171947 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 03 12:31:54.171947 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application result.read() Oct 03 12:31:54.171947 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 03 12:31:54.172970 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 03 12:31:54.172970 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 03 12:31:54.172970 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 03 12:31:54.172970 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 03 12:31:54.172970 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 03 12:31:54.172970 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 03 12:31:54.172970 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 03 12:31:54.172970 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 03 12:31:54.172970 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:54.172970 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 03 12:31:54.172970 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:54.172970 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:54.172970 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 03 12:31:54.172970 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 03 12:31:54.172970 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 03 12:31:54.172970 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 03 12:31:54.172970 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 03 12:31:54.172970 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application next(self.gen) Oct 03 12:31:54.174113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 03 12:31:54.174113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application yield resource Oct 03 12:31:54.174113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 03 12:31:54.174113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application next(self.gen) Oct 03 12:31:54.174113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 03 12:31:54.174113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self.session.rollback() Oct 03 12:31:54.174113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 03 12:31:54.174113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self.force_reraise() Oct 03 12:31:54.174113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 03 12:31:54.174113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise self.value Oct 03 12:31:54.174113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 03 12:31:54.174113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 03 12:31:54.174113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 03 12:31:54.174113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application session.commit() Oct 03 12:31:54.174113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 03 12:31:54.174113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 03 12:31:54.174113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 03 12:31:54.175193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._prepare_impl() Oct 03 12:31:54.175193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 03 12:31:54.175193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self.session.flush() Oct 03 12:31:54.175193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 03 12:31:54.175193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._flush(objects) Oct 03 12:31:54.175193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 03 12:31:54.175193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 03 12:31:54.175193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 03 12:31:54.175193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application compat.raise_( Oct 03 12:31:54.175193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 03 12:31:54.175193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception Oct 03 12:31:54.175193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 03 12:31:54.175193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application flush_context.execute() Oct 03 12:31:54.175193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 03 12:31:54.175193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rec.execute(self) Oct 03 12:31:54.175193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 03 12:31:54.175193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 03 12:31:54.176277 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 03 12:31:54.176277 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 03 12:31:54.176277 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 03 12:31:54.176277 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 03 12:31:54.176277 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 03 12:31:54.176277 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 03 12:31:54.176277 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 03 12:31:54.176277 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 03 12:31:54.176277 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 03 12:31:54.176277 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 03 12:31:54.176277 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 03 12:31:54.176277 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 03 12:31:54.176277 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 03 12:31:54.176277 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 03 12:31:54.176277 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 03 12:31:54.176277 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception Oct 03 12:31:54.177109 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 03 12:31:54.177109 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 03 12:31:54.177109 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 03 12:31:54.177109 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 03 12:31:54.177109 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 03 12:31:54.177109 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application result = self._query(query) Oct 03 12:31:54.177109 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 03 12:31:54.177109 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application conn.query(q) Oct 03 12:31:54.177109 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 03 12:31:54.177109 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 03 12:31:54.177109 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 03 12:31:54.177109 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application result.read() Oct 03 12:31:54.177109 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 03 12:31:54.177109 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 03 12:31:54.177109 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 03 12:31:54.177109 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 03 12:31:54.178830 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 03 12:31:54.178830 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 03 12:31:54.178830 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 03 12:31:54.178830 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 03 12:31:54.178830 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 03 12:31:54.178830 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 03 12:31:54.178830 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ee6c164e689d404481bef790683357fb', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 03 12:31:54.178830 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 03 12:31:54.178830 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:54.178830 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 12:31:54.178830 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:54.178830 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:54.178830 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:54.178830 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:54.178830 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:54.179732 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:54.179732 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:54.179732 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:54.179732 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:54.179732 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:54.179732 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:54.179732 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:54.179732 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 03 12:31:54.179732 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 03 12:31:54.179732 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:54.179732 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:54.179732 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 12:31:54.179732 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:54.179732 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 12:31:54.179732 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:54.179732 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 03 12:31:54.179732 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 03 12:31:54.180904 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 03 12:31:54.180904 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 03 12:31:54.180904 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 03 12:31:54.180904 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 03 12:31:54.180904 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 03 12:31:54.180904 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:54.182881 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 473/963] 10.222.0.33 () {60 vars in 1131 bytes} [Mon Oct 3 12:31:53 2022] POST /identity/v3/users => generated 160 bytes in 214 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:54.190776 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036a184-9b6a-4293-8cd8-e356669c3542 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:54.299821 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036a184-9b6a-4293-8cd8-e356669c3542 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:54.324341 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:54.336031 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aaf79f-6e9f-4347-9410-d9f79779258c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:54.336591 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aaf79f-6e9f-4347-9410-d9f79779258c None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:54.337211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aaf79f-6e9f-4347-9410-d9f79779258c None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:54.337490 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aaf79f-6e9f-4347-9410-d9f79779258c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:54.345199 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aaf79f-6e9f-4347-9410-d9f79779258c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:54.346631 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aaf79f-6e9f-4347-9410-d9f79779258c None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:54.358599 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-83aaf79f-6e9f-4347-9410-d9f79779258c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 03 12:31:54.358599 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:54.358599 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:54.358599 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:54.358599 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:54.358599 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:54.358599 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:54.358599 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:54.358599 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:54.358599 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:54.358599 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:54.358599 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:54.358599 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 12:31:54.358599 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 12:31:54.358599 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 12:31:54.358599 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 12:31:54.359713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:54.359713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:54.359713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 12:31:54.359713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:54.359713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 12:31:54.359713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:54.359713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:54.359713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:54.359713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:54.359713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:54.359713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:54.359713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:54.359713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:54.359713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:54.359713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:54.359713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:54.359713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:54.359713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:54.360690 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:54.360690 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:54.360690 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 12:31:54.360690 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 12:31:54.360690 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 12:31:54.360690 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 12:31:54.360690 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 12:31:54.360690 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 12:31:54.360690 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 03 12:31:54.360690 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:54.360690 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 491/964] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:31:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:54.368827 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2706a-e425-4537-aaf1-bf33160cc806 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:54.480155 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2706a-e425-4537-aaf1-bf33160cc806 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:54.481756 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:54.515988 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8137a399-bb4f-4fe2-9552-b09deff0e7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:54.517118 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137a399-bb4f-4fe2-9552-b09deff0e7bf None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:54.517992 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137a399-bb4f-4fe2-9552-b09deff0e7bf None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:54.518242 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137a399-bb4f-4fe2-9552-b09deff0e7bf None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:54.519006 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8137a399-bb4f-4fe2-9552-b09deff0e7bf None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:54.519335 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8137a399-bb4f-4fe2-9552-b09deff0e7bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:54.520972 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8137a399-bb4f-4fe2-9552-b09deff0e7bf None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:54.560239 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 474/965] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 12:31:54 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:55.052256 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1bb144-e01b-4383-81ef-277224020132 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:55.053220 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1bb144-e01b-4383-81ef-277224020132 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:55.054071 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1bb144-e01b-4383-81ef-277224020132 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:55.055562 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 492/966] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:55.060901 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e6206f-83b5-45a9-885d-95be1b304f5d None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:55.061191 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e6206f-83b5-45a9-885d-95be1b304f5d None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:55.061464 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e6206f-83b5-45a9-885d-95be1b304f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:55.081912 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-72e6206f-83b5-45a9-885d-95be1b304f5d None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:55.083553 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-72e6206f-83b5-45a9-885d-95be1b304f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:55.153275 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 475/967] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:31:55.309256 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5f9eb-b662-49f5-82b1-d12a07ce3e03 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:55.309547 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5f9eb-b662-49f5-82b1-d12a07ce3e03 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:55.309725 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5f9eb-b662-49f5-82b1-d12a07ce3e03 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:55.311079 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 493/968] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:55.316319 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205cc034-459c-444f-92c3-dd0f12e5cc83 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:55.316605 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205cc034-459c-444f-92c3-dd0f12e5cc83 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:55.316876 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205cc034-459c-444f-92c3-dd0f12e5cc83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:55.335649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-205cc034-459c-444f-92c3-dd0f12e5cc83 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:55.336812 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-205cc034-459c-444f-92c3-dd0f12e5cc83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:55.404007 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 476/969] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:31:55.423376 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5f9eb-b662-49f5-82b1-d12a07ce3e03 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:55.528098 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5f9eb-b662-49f5-82b1-d12a07ce3e03 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:55.560941 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3285ea-5407-4b80-a3ee-c7a58fd82641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:55.561495 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3285ea-5407-4b80-a3ee-c7a58fd82641 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:55.561769 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3285ea-5407-4b80-a3ee-c7a58fd82641 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:55.562035 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3285ea-5407-4b80-a3ee-c7a58fd82641 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:55.562605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3285ea-5407-4b80-a3ee-c7a58fd82641 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:55.563979 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3285ea-5407-4b80-a3ee-c7a58fd82641 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:55.573383 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-ef3285ea-5407-4b80-a3ee-c7a58fd82641 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 12:31:55.573383 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:55.573383 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 12:31:55.573383 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 12:31:55.573383 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 12:31:55.573383 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 12:31:55.573383 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 12:31:55.573383 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:31:55.573383 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 12:31:55.573383 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:55.573383 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 12:31:55.573383 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:55.573383 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:55.573383 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:55.573383 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:55.573383 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:55.573383 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:55.574385 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:55.574385 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:55.574385 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:55.574385 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:55.574385 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:55.574385 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:55.574385 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 12:31:55.574385 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 12:31:55.574385 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 12:31:55.574385 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 12:31:55.574385 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:55.574385 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:55.574385 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:55.574385 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:55.574385 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:55.574385 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:55.576384 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:55.576384 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:55.576384 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:55.576384 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:55.576384 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:55.576384 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:55.576384 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:55.576384 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:55.576384 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:55.576384 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:55.576384 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 12:31:55.576384 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:31:55.576384 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 12:31:55.576384 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:55.576384 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 494/970] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Oct 3 12:31:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:55.580564 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205cc034-459c-444f-92c3-dd0f12e5cc83 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:55.603655 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:55.678034 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3285ea-5407-4b80-a3ee-c7a58fd82641 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:55.681353 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205cc034-459c-444f-92c3-dd0f12e5cc83 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:55.716979 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fb27e7-0fbf-4823-8247-a951c00359a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:55.717598 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb27e7-0fbf-4823-8247-a951c00359a2 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:55.717876 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb27e7-0fbf-4823-8247-a951c00359a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:55.718144 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb27e7-0fbf-4823-8247-a951c00359a2 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:55.719024 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fb27e7-0fbf-4823-8247-a951c00359a2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:55.719340 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fb27e7-0fbf-4823-8247-a951c00359a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:55.720708 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fb27e7-0fbf-4823-8247-a951c00359a2 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:55.725464 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 477/971] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Oct 3 12:31:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:31:55.731038 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fb27e7-0fbf-4823-8247-a951c00359a2 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:55.734258 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:55.793594 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3285ea-5407-4b80-a3ee-c7a58fd82641 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:55.824487 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:55.827970 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c78c39-42f0-4f16-a7f8-66e22ef625d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:55.828629 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c78c39-42f0-4f16-a7f8-66e22ef625d6 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:55.828908 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c78c39-42f0-4f16-a7f8-66e22ef625d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:55.829177 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c78c39-42f0-4f16-a7f8-66e22ef625d6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:55.841067 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c78c39-42f0-4f16-a7f8-66e22ef625d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:31:55.844206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fb27e7-0fbf-4823-8247-a951c00359a2 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:55.846811 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-00c78c39-42f0-4f16-a7f8-66e22ef625d6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 12:31:55.846811 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:55.846811 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:55.846811 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:55.846811 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:55.846811 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:55.846811 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:55.846811 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:55.846811 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:55.846811 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:55.846811 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:55.846811 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:55.846811 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:31:55.846811 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:31:55.846811 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:31:55.846811 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:31:55.847939 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:31:55.847939 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:31:55.847939 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:31:55.847939 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:31:55.847939 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:55.847939 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:55.847939 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:55.847939 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:55.847939 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:55.847939 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:55.847939 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:55.847939 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:55.847939 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:55.847939 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:55.847939 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:55.847939 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:55.849095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:55.849095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:55.849095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:55.849095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:55.849095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:31:55.849095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:31:55.849095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:31:55.849095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:31:55.849095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:31:55.849095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:31:55.849095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 12:31:55.849095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:55.849095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 495/972] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Oct 3 12:31:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:31:55.854370 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c78c39-42f0-4f16-a7f8-66e22ef625d6 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:55.887590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6915ac-84b8-4b2c-b0a2-af8c24e1515d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:55.888468 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6915ac-84b8-4b2c-b0a2-af8c24e1515d None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:55.888886 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6915ac-84b8-4b2c-b0a2-af8c24e1515d None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:55.889397 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6915ac-84b8-4b2c-b0a2-af8c24e1515d None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:55.890278 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6915ac-84b8-4b2c-b0a2-af8c24e1515d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:55.890575 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6915ac-84b8-4b2c-b0a2-af8c24e1515d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:55.892011 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6915ac-84b8-4b2c-b0a2-af8c24e1515d None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:55.898765 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 478/973] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Oct 3 12:31:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:31:55.904508 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6915ac-84b8-4b2c-b0a2-af8c24e1515d None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:55.960345 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c78c39-42f0-4f16-a7f8-66e22ef625d6 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:55.969645 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:55.996771 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a92393f-4e97-4088-81d6-b368187a1a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:55.997431 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a92393f-4e97-4088-81d6-b368187a1a2d None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:55.997758 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a92393f-4e97-4088-81d6-b368187a1a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:55.998031 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a92393f-4e97-4088-81d6-b368187a1a2d None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:55.998689 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a92393f-4e97-4088-81d6-b368187a1a2d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:55.998993 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a92393f-4e97-4088-81d6-b368187a1a2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:56.000375 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a92393f-4e97-4088-81d6-b368187a1a2d None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:56.004721 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6915ac-84b8-4b2c-b0a2-af8c24e1515d None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:56.009150 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 496/974] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Oct 3 12:31:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:31:56.010311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0a81c-2b6e-4686-9c02-46aa37d7b7ef None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:56.010596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0a81c-2b6e-4686-9c02-46aa37d7b7ef None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:56.010871 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0a81c-2b6e-4686-9c02-46aa37d7b7ef None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:56.012234 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 497/975] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:56.016274 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece0a81c-2b6e-4686-9c02-46aa37d7b7ef None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:56.040935 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2243826d-f5d5-40df-be0f-2a4be2117679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:56.041547 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243826d-f5d5-40df-be0f-2a4be2117679 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:56.041825 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243826d-f5d5-40df-be0f-2a4be2117679 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:56.042104 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243826d-f5d5-40df-be0f-2a4be2117679 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:56.042804 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2243826d-f5d5-40df-be0f-2a4be2117679 None admin] RBAC: Adding query filter params (interface=admin, service_id=570bb19fd12e48859e3967f3eaa7a0dc, region_id=RegionOne) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:56.043110 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2243826d-f5d5-40df-be0f-2a4be2117679 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:56.044510 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2243826d-f5d5-40df-be0f-2a4be2117679 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:56.049573 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 479/976] 10.222.0.33 () {56 vars in 1249 bytes} [Mon Oct 3 12:31:55 2022] GET /identity/v3/endpoints?service_id=570bb19fd12e48859e3967f3eaa7a0dc&interface=admin®ion_id=RegionOne => generated 193 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:31:56.050697 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6704849f-311f-4936-8d96-f1ca0059a69a None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:56.050979 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6704849f-311f-4936-8d96-f1ca0059a69a None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:56.051251 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6704849f-311f-4936-8d96-f1ca0059a69a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:56.069200 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-6704849f-311f-4936-8d96-f1ca0059a69a None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:56.070374 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-6704849f-311f-4936-8d96-f1ca0059a69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:56.078256 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:56.123862 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece0a81c-2b6e-4686-9c02-46aa37d7b7ef None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:56.139403 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 480/977] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:31:56.140259 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6704849f-311f-4936-8d96-f1ca0059a69a None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:56.164148 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b42b096-a1a1-4a42-9496-f361afb876df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:56.164777 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42b096-a1a1-4a42-9496-f361afb876df None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:56.165063 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42b096-a1a1-4a42-9496-f361afb876df None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:56.165332 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42b096-a1a1-4a42-9496-f361afb876df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:56.175532 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b42b096-a1a1-4a42-9496-f361afb876df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:56.176958 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b42b096-a1a1-4a42-9496-f361afb876df None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:56.192273 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-4b42b096-a1a1-4a42-9496-f361afb876df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:56.192273 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:56.192273 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:56.192273 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:56.192273 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:56.192273 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:56.192273 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:56.192273 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:56.192273 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:56.192273 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:56.192273 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:56.192273 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:56.192273 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:56.192273 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:56.192273 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:56.192273 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:56.193915 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:56.193915 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:56.193915 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:56.193915 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:56.193915 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:56.193915 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:56.193915 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:56.193915 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:56.193915 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:56.193915 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:56.193915 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:56.193915 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:56.193915 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:56.193915 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:56.193915 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:56.193915 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:56.193915 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:56.194821 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:56.194821 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:56.194821 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:56.194821 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 498/978] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:56.198947 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b42b096-a1a1-4a42-9496-f361afb876df None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:56.227284 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:56.260788 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6704849f-311f-4936-8d96-f1ca0059a69a None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:56.297070 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be9f367-b524-4d1e-be21-97e466252018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:56.297676 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9f367-b524-4d1e-be21-97e466252018 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:56.297953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9f367-b524-4d1e-be21-97e466252018 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:56.298224 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9f367-b524-4d1e-be21-97e466252018 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:56.298821 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be9f367-b524-4d1e-be21-97e466252018 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:56.300257 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be9f367-b524-4d1e-be21-97e466252018 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:56.315350 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 481/979] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Oct 3 12:31:56 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:31:56.329742 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b42b096-a1a1-4a42-9496-f361afb876df None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:56.370015 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e8894e-12db-4cdc-802a-3fe0edbb8d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:56.370650 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e8894e-12db-4cdc-802a-3fe0edbb8d23 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:56.370934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e8894e-12db-4cdc-802a-3fe0edbb8d23 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:56.371211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e8894e-12db-4cdc-802a-3fe0edbb8d23 None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:56.371907 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e8894e-12db-4cdc-802a-3fe0edbb8d23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:56.372217 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e8894e-12db-4cdc-802a-3fe0edbb8d23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:56.373610 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e8894e-12db-4cdc-802a-3fe0edbb8d23 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:56.387081 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 499/980] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:31:56.392817 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be9f367-b524-4d1e-be21-97e466252018 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:56.402837 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:56.412748 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e8894e-12db-4cdc-802a-3fe0edbb8d23 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:56.508424 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:56.509959 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be9f367-b524-4d1e-be21-97e466252018 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:56.530001 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e8894e-12db-4cdc-802a-3fe0edbb8d23 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:56.546022 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b89679-74d3-4837-80a0-64f230ea475b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:56.546755 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b89679-74d3-4837-80a0-64f230ea475b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:56.547039 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b89679-74d3-4837-80a0-64f230ea475b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:56.547354 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b89679-74d3-4837-80a0-64f230ea475b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:56.556753 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b89679-74d3-4837-80a0-64f230ea475b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:56.558189 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b89679-74d3-4837-80a0-64f230ea475b None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:56.566873 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca99b11f-351a-4a21-afd1-ee65e434f2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:56.567518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99b11f-351a-4a21-afd1-ee65e434f2e9 None admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:56.567806 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99b11f-351a-4a21-afd1-ee65e434f2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:56.568079 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99b11f-351a-4a21-afd1-ee65e434f2e9 None admin] PATH_INFO: `/v3/services` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:56.568992 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca99b11f-351a-4a21-afd1-ee65e434f2e9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:56.570343 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca99b11f-351a-4a21-afd1-ee65e434f2e9 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:56.570651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-88b89679-74d3-4837-80a0-64f230ea475b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 12:31:56.570651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:56.570651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:56.570651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:56.570651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:56.570651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:56.570651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:56.570651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:56.570651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:56.570651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:56.570651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:56.570651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:56.570651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 12:31:56.570651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 12:31:56.570651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 12:31:56.570651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 12:31:56.571652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:56.571652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:56.571652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 12:31:56.571652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:56.571652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 12:31:56.571652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:31:56.571652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:56.571652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:56.571652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:56.571652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:56.571652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:56.571652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:56.571652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:56.571652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:56.571652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:56.571652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:56.571652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:56.571652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:56.572576 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:56.572576 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:56.572576 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 12:31:56.572576 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 12:31:56.572576 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 12:31:56.572576 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 12:31:56.572576 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 12:31:56.572576 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 12:31:56.572576 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 12:31:56.572576 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:56.575474 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 482/981] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Oct 3 12:31:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:56.577385 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b89679-74d3-4837-80a0-64f230ea475b None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:56.644562 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:56.651702 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 500/982] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Oct 3 12:31:56 2022] POST /identity/v3/services => generated 242 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:31:56.690654 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b89679-74d3-4837-80a0-64f230ea475b None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:56.736514 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf8cf44-3e71-4784-950c-825e22e091c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:56.737139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8cf44-3e71-4784-950c-825e22e091c9 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:56.737414 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8cf44-3e71-4784-950c-825e22e091c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:56.737682 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8cf44-3e71-4784-950c-825e22e091c9 None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:56.738415 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf8cf44-3e71-4784-950c-825e22e091c9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:56.738716 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf8cf44-3e71-4784-950c-825e22e091c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:56.740304 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf8cf44-3e71-4784-950c-825e22e091c9 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:56.759843 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 483/983] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Oct 3 12:31:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:56.765376 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca99b11f-351a-4a21-afd1-ee65e434f2e9 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:56.875077 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca99b11f-351a-4a21-afd1-ee65e434f2e9 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:56.914998 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678f4f8f-3c8f-4749-947c-5af47922ab59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:56.914998 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678f4f8f-3c8f-4749-947c-5af47922ab59 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:56.914998 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678f4f8f-3c8f-4749-947c-5af47922ab59 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:56.914998 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678f4f8f-3c8f-4749-947c-5af47922ab59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:56.923096 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:56.978782 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678f4f8f-3c8f-4749-947c-5af47922ab59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:56.980339 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678f4f8f-3c8f-4749-947c-5af47922ab59 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:57.079746 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 501/984] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:31:56 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 12:31:57.085309 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf8cf44-3e71-4784-950c-825e22e091c9 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:57.188378 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf8cf44-3e71-4784-950c-825e22e091c9 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:57.210037 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:57.226991 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156e9c4-6b1a-44e7-b419-46701f8964ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:57.227732 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156e9c4-6b1a-44e7-b419-46701f8964ac None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:57.230222 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156e9c4-6b1a-44e7-b419-46701f8964ac None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:57.230222 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156e9c4-6b1a-44e7-b419-46701f8964ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:57.241752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156e9c4-6b1a-44e7-b419-46701f8964ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:57.243210 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156e9c4-6b1a-44e7-b419-46701f8964ac None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:57.265615 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-8156e9c4-6b1a-44e7-b419-46701f8964ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:57.265615 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:57.265615 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:57.265615 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:57.265615 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:57.265615 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:57.265615 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:57.265615 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:57.265615 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:57.265615 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:57.265615 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:57.265615 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:57.265615 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:31:57.265615 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:31:57.265615 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:31:57.265615 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:31:57.266571 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:57.266571 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:57.266571 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:57.266571 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:57.266571 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:57.266571 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:57.266571 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:57.266571 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:57.266571 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:57.266571 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:57.266571 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:57.266571 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:57.266571 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:57.266571 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:57.266571 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:57.266571 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:57.266571 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:31:57.272675 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:31:57.272675 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:31:57.272675 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:57.272675 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 484/985] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:31:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:57.274293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678f4f8f-3c8f-4749-947c-5af47922ab59 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:57.318821 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:57.373993 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678f4f8f-3c8f-4749-947c-5af47922ab59 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:57.407579 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f250dbcc-789f-4be7-8367-dc8885016fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:57.408185 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250dbcc-789f-4be7-8367-dc8885016fcc None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:57.408457 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250dbcc-789f-4be7-8367-dc8885016fcc None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:57.408715 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250dbcc-789f-4be7-8367-dc8885016fcc None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:57.409343 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f250dbcc-789f-4be7-8367-dc8885016fcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:57.409641 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f250dbcc-789f-4be7-8367-dc8885016fcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:57.410974 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f250dbcc-789f-4be7-8367-dc8885016fcc None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:57.422885 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 502/986] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:31:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:57.428497 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156e9c4-6b1a-44e7-b419-46701f8964ac None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:57.527861 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156e9c4-6b1a-44e7-b419-46701f8964ac None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:57.564198 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d9a798-cf26-4dfd-a012-b1ddb85f2d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:57.564832 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9a798-cf26-4dfd-a012-b1ddb85f2d78 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:57.565106 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9a798-cf26-4dfd-a012-b1ddb85f2d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:57.565372 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9a798-cf26-4dfd-a012-b1ddb85f2d78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:57.572873 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d9a798-cf26-4dfd-a012-b1ddb85f2d78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:57.574295 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d9a798-cf26-4dfd-a012-b1ddb85f2d78 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:57.585391 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-a4d9a798-cf26-4dfd-a012-b1ddb85f2d78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:57.585391 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:57.585391 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:57.585391 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:57.585391 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:57.585391 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:57.585391 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:57.585391 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:57.585391 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:57.585391 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:57.585391 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:57.585391 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:57.585391 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 12:31:57.585391 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 12:31:57.585391 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 12:31:57.586282 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 12:31:57.586282 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:57.586282 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:57.586282 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:57.586282 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:57.586282 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:57.586282 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:57.586282 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:57.586282 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:57.586282 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:57.586282 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:57.586282 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:57.586282 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:57.586282 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:57.586282 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:57.586282 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:57.586282 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:57.587755 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 12:31:57.587755 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 12:31:57.587755 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 12:31:57.587755 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 12:31:57.587755 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 12:31:57.587755 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 12:31:57.587755 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:31:57.587755 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:31:57.587755 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 485/987] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:31:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:31:57.592253 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f250dbcc-789f-4be7-8367-dc8885016fcc None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:57.690492 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f250dbcc-789f-4be7-8367-dc8885016fcc None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:57.692000 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:57.723849 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c584b812-087a-4d9c-9c7a-3fd06894892c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:57.725076 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584b812-087a-4d9c-9c7a-3fd06894892c None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:57.725369 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584b812-087a-4d9c-9c7a-3fd06894892c None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:57.725719 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584b812-087a-4d9c-9c7a-3fd06894892c None admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:57.726392 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c584b812-087a-4d9c-9c7a-3fd06894892c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:57.726695 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c584b812-087a-4d9c-9c7a-3fd06894892c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:57.728119 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c584b812-087a-4d9c-9c7a-3fd06894892c None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:57.744886 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 503/988] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 12:31:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:57.746667 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d9a798-cf26-4dfd-a012-b1ddb85f2d78 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:57.779553 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:57.847631 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d9a798-cf26-4dfd-a012-b1ddb85f2d78 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:57.885562 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b7638c-9ce6-4624-a4bf-16b80dded3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:57.886162 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7638c-9ce6-4624-a4bf-16b80dded3f9 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:57.886438 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7638c-9ce6-4624-a4bf-16b80dded3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:57.886708 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7638c-9ce6-4624-a4bf-16b80dded3f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:57.887458 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b7638c-9ce6-4624-a4bf-16b80dded3f9 None admin] RBAC: Adding query filter params (role.id=a7c026bed2224838b2a71641eb61b9d0, scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7, user.id=2e6e4387a7c24bbc82c22eefe2a3d138) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:57.887764 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b7638c-9ce6-4624-a4bf-16b80dded3f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:57.889174 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b7638c-9ce6-4624-a4bf-16b80dded3f9 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:57.894919 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 486/989] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:31:57 2022] GET /identity/v3/role_assignments?user.id=2e6e4387a7c24bbc82c22eefe2a3d138&scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7&role.id=a7c026bed2224838b2a71641eb61b9d0 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:31:58.548247 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6bb950-aef4-4e08-8d6d-2e75b43ef359 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:58.548587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6bb950-aef4-4e08-8d6d-2e75b43ef359 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:58.548765 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6bb950-aef4-4e08-8d6d-2e75b43ef359 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:58.550251 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 504/990] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:58.555339 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa22ee-7557-43e9-b43d-6d639fe41b92 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:58.555776 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa22ee-7557-43e9-b43d-6d639fe41b92 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:58.555938 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa22ee-7557-43e9-b43d-6d639fe41b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:58.574661 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-a3aa22ee-7557-43e9-b43d-6d639fe41b92 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:58.575869 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a3aa22ee-7557-43e9-b43d-6d639fe41b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:58.645684 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 487/991] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:58 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:31:58.851785 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a77cc96-1304-4680-9e10-d4004fb0a0d4 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:58.852317 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a77cc96-1304-4680-9e10-d4004fb0a0d4 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:58.852317 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a77cc96-1304-4680-9e10-d4004fb0a0d4 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:58.853726 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 505/992] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:31:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:31:58.858808 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576edd57-fa8d-4bb9-b975-3d8d5784506d None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:58.859096 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576edd57-fa8d-4bb9-b975-3d8d5784506d None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:58.859391 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576edd57-fa8d-4bb9-b975-3d8d5784506d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:58.878002 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-576edd57-fa8d-4bb9-b975-3d8d5784506d None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:31:58.879167 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-576edd57-fa8d-4bb9-b975-3d8d5784506d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:31:58.899782 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:58.914163 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a77cc96-1304-4680-9e10-d4004fb0a0d4 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:58.949858 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 488/993] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:31:58 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:31:59.024783 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a77cc96-1304-4680-9e10-d4004fb0a0d4 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:59.058279 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f673b4-10aa-4339-bf88-f382ac5666bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:59.059133 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f673b4-10aa-4339-bf88-f382ac5666bf None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:59.059593 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f673b4-10aa-4339-bf88-f382ac5666bf None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:59.060019 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f673b4-10aa-4339-bf88-f382ac5666bf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:59.060737 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f673b4-10aa-4339-bf88-f382ac5666bf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:59.062211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f673b4-10aa-4339-bf88-f382ac5666bf None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:59.071037 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-69f673b4-10aa-4339-bf88-f382ac5666bf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 12:31:59.071037 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:59.071037 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 12:31:59.071037 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 12:31:59.071037 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 12:31:59.071037 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 12:31:59.071037 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 12:31:59.071037 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:31:59.071037 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 12:31:59.071037 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:59.071037 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 12:31:59.071037 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:59.071037 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:59.071037 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:59.071037 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:59.071037 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:59.071037 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:59.072265 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:59.072265 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:59.072265 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:59.072265 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:59.072265 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:59.072265 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:59.072265 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 12:31:59.072265 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 12:31:59.072265 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 12:31:59.072265 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 12:31:59.072265 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:59.072265 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:59.072265 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:59.072265 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:59.072265 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:59.072265 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:59.073303 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:59.073303 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:59.073303 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:59.073303 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:59.073303 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:59.073303 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:59.073303 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:59.073303 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:59.073303 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:59.073303 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:59.073303 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 12:31:59.073303 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:31:59.073303 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 12:31:59.073303 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:59.075790 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 506/994] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Oct 3 12:31:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:31:59.081316 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576edd57-fa8d-4bb9-b975-3d8d5784506d None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:59.183123 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576edd57-fa8d-4bb9-b975-3d8d5784506d None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:59.215618 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f673b4-10aa-4339-bf88-f382ac5666bf None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:59.217919 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64770872-b13f-46e5-8782-c09071d03d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:59.218933 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64770872-b13f-46e5-8782-c09071d03d89 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:59.219220 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64770872-b13f-46e5-8782-c09071d03d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:59.219527 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64770872-b13f-46e5-8782-c09071d03d89 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:59.220153 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64770872-b13f-46e5-8782-c09071d03d89 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:59.220455 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64770872-b13f-46e5-8782-c09071d03d89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:59.221806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64770872-b13f-46e5-8782-c09071d03d89 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:59.238374 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 489/995] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Oct 3 12:31:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:31:59.242642 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64770872-b13f-46e5-8782-c09071d03d89 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:59.251588 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:59.270343 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:59.330096 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f673b4-10aa-4339-bf88-f382ac5666bf None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:59.360571 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64770872-b13f-46e5-8782-c09071d03d89 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:59.367557 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f51feeb-1ecd-480a-aa02-06c11aac2b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:59.368382 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51feeb-1ecd-480a-aa02-06c11aac2b65 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:59.368495 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51feeb-1ecd-480a-aa02-06c11aac2b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:59.368713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51feeb-1ecd-480a-aa02-06c11aac2b65 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:59.369279 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f51feeb-1ecd-480a-aa02-06c11aac2b65 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:59.370702 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f51feeb-1ecd-480a-aa02-06c11aac2b65 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:59.380129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-0f51feeb-1ecd-480a-aa02-06c11aac2b65 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 03 12:31:59.380129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:59.380129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 12:31:59.380129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 12:31:59.380129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 12:31:59.380129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 12:31:59.380129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 12:31:59.380129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:31:59.380129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 03 12:31:59.380129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:59.380129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 12:31:59.380129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:59.380129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:31:59.380129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:31:59.380129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:31:59.380129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:31:59.380129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:31:59.381860 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:31:59.381860 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:31:59.381860 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:31:59.381860 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:31:59.381860 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:31:59.381860 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:31:59.381860 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 12:31:59.381860 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 12:31:59.381860 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 12:31:59.381860 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 12:31:59.381860 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:31:59.381860 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:31:59.381860 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:31:59.381860 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:31:59.381860 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:31:59.381860 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:31:59.383280 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:31:59.383280 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:31:59.383280 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:31:59.383280 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:31:59.383280 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:31:59.383280 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:31:59.383280 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:31:59.383280 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:31:59.383280 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:31:59.383280 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:31:59.383280 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 12:31:59.383280 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:31:59.383280 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 03 12:31:59.383280 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:31:59.383280 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 507/996] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:31:59 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:31:59.387583 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f51feeb-1ecd-480a-aa02-06c11aac2b65 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:59.400249 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aab555-ff41-4d97-8f80-98e3539caa42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:59.400862 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aab555-ff41-4d97-8f80-98e3539caa42 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:59.401136 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aab555-ff41-4d97-8f80-98e3539caa42 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:59.401405 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aab555-ff41-4d97-8f80-98e3539caa42 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:59.402024 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aab555-ff41-4d97-8f80-98e3539caa42 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:59.402323 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aab555-ff41-4d97-8f80-98e3539caa42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:59.403713 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aab555-ff41-4d97-8f80-98e3539caa42 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:59.413697 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 490/997] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Oct 3 12:31:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:31:59.415607 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aab555-ff41-4d97-8f80-98e3539caa42 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:59.486463 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:59.497419 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f51feeb-1ecd-480a-aa02-06c11aac2b65 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:59.503940 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:31:59.519510 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aab555-ff41-4d97-8f80-98e3539caa42 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:59.533413 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb0f5e-c9d3-44ec-96ea-914c0d3b5952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:59.534019 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb0f5e-c9d3-44ec-96ea-914c0d3b5952 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:59.534300 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb0f5e-c9d3-44ec-96ea-914c0d3b5952 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:59.534569 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb0f5e-c9d3-44ec-96ea-914c0d3b5952 None admin] PATH_INFO: `/v3/services` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:59.535184 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addb0f5e-c9d3-44ec-96ea-914c0d3b5952 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:59.535514 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addb0f5e-c9d3-44ec-96ea-914c0d3b5952 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:59.537033 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addb0f5e-c9d3-44ec-96ea-914c0d3b5952 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:59.546810 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 508/998] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Oct 3 12:31:59 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:31:59.547723 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb0f5e-c9d3-44ec-96ea-914c0d3b5952 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:59.563188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb9fdd-a99c-48db-9d1d-e6f557e5c351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:59.563845 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb9fdd-a99c-48db-9d1d-e6f557e5c351 None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:59.564123 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb9fdd-a99c-48db-9d1d-e6f557e5c351 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:59.564399 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb9fdd-a99c-48db-9d1d-e6f557e5c351 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:59.565064 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdb9fdd-a99c-48db-9d1d-e6f557e5c351 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:59.566421 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdb9fdd-a99c-48db-9d1d-e6f557e5c351 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:59.641801 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 491/999] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:31:59 2022] POST /identity/v3/endpoints => generated 325 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:31:59.649347 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb0f5e-c9d3-44ec-96ea-914c0d3b5952 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:59.696598 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e731008-d5c4-43e9-b00d-15916453a128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:59.697227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e731008-d5c4-43e9-b00d-15916453a128 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:59.697504 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e731008-d5c4-43e9-b00d-15916453a128 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:59.697772 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e731008-d5c4-43e9-b00d-15916453a128 None admin] PATH_INFO: `/v3/services` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:59.698415 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e731008-d5c4-43e9-b00d-15916453a128 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:59.698715 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e731008-d5c4-43e9-b00d-15916453a128 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:59.700169 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e731008-d5c4-43e9-b00d-15916453a128 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:59.710943 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 509/1000] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Oct 3 12:31:59 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:31:59.712714 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb9fdd-a99c-48db-9d1d-e6f557e5c351 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:59.831856 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb9fdd-a99c-48db-9d1d-e6f557e5c351 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:31:59.867553 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e02704-3dac-4302-bc01-64361dd48c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:31:59.868178 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e02704-3dac-4302-bc01-64361dd48c49 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:31:59.868450 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e02704-3dac-4302-bc01-64361dd48c49 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:31:59.868718 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e02704-3dac-4302-bc01-64361dd48c49 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:31:59.869409 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e02704-3dac-4302-bc01-64361dd48c49 None admin] RBAC: Adding query filter params (interface=public, service_id=39c8d17a4c40467390355b7e8081bb44, region_id=RegionOne) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:31:59.869733 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e02704-3dac-4302-bc01-64361dd48c49 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:31:59.871082 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e02704-3dac-4302-bc01-64361dd48c49 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:31:59.876368 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 492/1001] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Oct 3 12:31:59 2022] GET /identity/v3/endpoints?service_id=39c8d17a4c40467390355b7e8081bb44&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:31:59.882158 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e731008-d5c4-43e9-b00d-15916453a128 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:31:59.899014 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:00.005579 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e731008-d5c4-43e9-b00d-15916453a128 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:00.040458 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a922b2aa-ba41-4085-b92c-f438cd674a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:00.041066 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a922b2aa-ba41-4085-b92c-f438cd674a63 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:00.041342 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a922b2aa-ba41-4085-b92c-f438cd674a63 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:00.041609 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a922b2aa-ba41-4085-b92c-f438cd674a63 None admin] PATH_INFO: `/v3/services` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:00.042201 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a922b2aa-ba41-4085-b92c-f438cd674a63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:00.043567 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a922b2aa-ba41-4085-b92c-f438cd674a63 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:00.049855 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 510/1002] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Oct 3 12:31:59 2022] GET /identity/v3/services => generated 1727 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 12:32:00.220884 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416af4d-f2c7-4e67-a3a7-1ca91359a287 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:00.221081 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416af4d-f2c7-4e67-a3a7-1ca91359a287 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:00.221353 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416af4d-f2c7-4e67-a3a7-1ca91359a287 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:00.222709 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 493/1003] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:32:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:32:00.234818 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e4209-008f-4c3b-9b23-0e1aaecc94e3 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:00.235105 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e4209-008f-4c3b-9b23-0e1aaecc94e3 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:00.236633 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e4209-008f-4c3b-9b23-0e1aaecc94e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:00.257462 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-021e4209-008f-4c3b-9b23-0e1aaecc94e3 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:32:00.258661 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-021e4209-008f-4c3b-9b23-0e1aaecc94e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:32:00.330912 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 511/1004] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:32:00 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:32:00.627613 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a416af4d-f2c7-4e67-a3a7-1ca91359a287 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:00.639613 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:00.731650 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a416af4d-f2c7-4e67-a3a7-1ca91359a287 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:00.769198 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b6e279-cc41-46f3-a144-db3f178ec1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:00.769828 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6e279-cc41-46f3-a144-db3f178ec1a8 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:00.770105 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6e279-cc41-46f3-a144-db3f178ec1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:00.770375 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6e279-cc41-46f3-a144-db3f178ec1a8 None admin] PATH_INFO: `/v3/services/image` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:00.770946 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b6e279-cc41-46f3-a144-db3f178ec1a8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:00.772461 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b6e279-cc41-46f3-a144-db3f178ec1a8 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:00.781469 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-91b6e279-cc41-46f3-a144-db3f178ec1a8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 03 12:32:00.781469 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:32:00.781469 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 12:32:00.781469 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 12:32:00.781469 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 12:32:00.781469 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 12:32:00.781469 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 12:32:00.781469 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:32:00.781469 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 03 12:32:00.781469 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:32:00.781469 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 12:32:00.781469 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:32:00.781469 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:32:00.781469 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:32:00.781469 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:32:00.781469 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:32:00.781469 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:32:00.782458 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:32:00.782458 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:32:00.782458 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:32:00.782458 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:32:00.782458 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:32:00.782458 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:32:00.782458 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 12:32:00.782458 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 12:32:00.782458 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 12:32:00.782458 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 12:32:00.782458 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:32:00.782458 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:32:00.782458 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:32:00.782458 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:32:00.782458 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:32:00.782458 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:32:00.784357 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:32:00.784357 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:32:00.784357 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:32:00.784357 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:32:00.784357 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:32:00.784357 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:32:00.784357 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:32:00.784357 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:32:00.784357 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:32:00.784357 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:32:00.784357 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 12:32:00.784357 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:32:00.784357 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 03 12:32:00.784357 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:32:00.784357 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 494/1005] 10.222.0.33 () {56 vars in 1105 bytes} [Mon Oct 3 12:32:00 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:32:00.789149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e4209-008f-4c3b-9b23-0e1aaecc94e3 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:00.799307 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:00.887991 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e4209-008f-4c3b-9b23-0e1aaecc94e3 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:00.921561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fcb571-68fa-45fc-bb96-bc8d20e1426d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:00.922232 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fcb571-68fa-45fc-bb96-bc8d20e1426d None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:00.922515 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fcb571-68fa-45fc-bb96-bc8d20e1426d None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:00.922784 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fcb571-68fa-45fc-bb96-bc8d20e1426d None admin] PATH_INFO: `/v3/services` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:00.923431 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fcb571-68fa-45fc-bb96-bc8d20e1426d None admin] RBAC: Adding query filter params (name=image) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:00.923739 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fcb571-68fa-45fc-bb96-bc8d20e1426d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:00.925080 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fcb571-68fa-45fc-bb96-bc8d20e1426d None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:00.934864 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 512/1006] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:32:00 2022] GET /identity/v3/services?name=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:00.936197 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b6e279-cc41-46f3-a144-db3f178ec1a8 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:01.013749 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:01.043761 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b6e279-cc41-46f3-a144-db3f178ec1a8 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:01.077844 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59abe9-5c0f-4329-9bb6-201cf0ba56a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:01.078449 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59abe9-5c0f-4329-9bb6-201cf0ba56a9 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:01.078734 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59abe9-5c0f-4329-9bb6-201cf0ba56a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:01.079004 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59abe9-5c0f-4329-9bb6-201cf0ba56a9 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:01.079650 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d59abe9-5c0f-4329-9bb6-201cf0ba56a9 None admin] RBAC: Adding query filter params (type=image) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:01.079952 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d59abe9-5c0f-4329-9bb6-201cf0ba56a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:01.081300 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d59abe9-5c0f-4329-9bb6-201cf0ba56a9 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:01.087476 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 495/1007] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:32:00 2022] GET /identity/v3/services?type=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:01.891557 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f4a41-0a40-4e2b-a806-27b6237ffb66 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:01.892152 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f4a41-0a40-4e2b-a806-27b6237ffb66 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:01.892152 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f4a41-0a40-4e2b-a806-27b6237ffb66 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:01.893573 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 513/1008] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:32:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:32:01.899288 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8622d6-8c32-4f31-94d8-d13ec384f9c5 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:01.899735 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8622d6-8c32-4f31-94d8-d13ec384f9c5 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:01.899918 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8622d6-8c32-4f31-94d8-d13ec384f9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:01.918757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-0b8622d6-8c32-4f31-94d8-d13ec384f9c5 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:32:01.919969 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-0b8622d6-8c32-4f31-94d8-d13ec384f9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:32:02.009585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 496/1009] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:32:01 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:32:02.236688 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987f208-b05f-44ce-9246-6e6a871edd20 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:02.236688 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987f208-b05f-44ce-9246-6e6a871edd20 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:02.236945 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987f208-b05f-44ce-9246-6e6a871edd20 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:02.238302 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 514/1010] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:32:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:32:02.243609 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803883f4-1d7f-4016-8e43-51bbc07b09ce None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:02.243891 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803883f4-1d7f-4016-8e43-51bbc07b09ce None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:02.246983 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803883f4-1d7f-4016-8e43-51bbc07b09ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:02.264515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-803883f4-1d7f-4016-8e43-51bbc07b09ce None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:32:02.265681 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-803883f4-1d7f-4016-8e43-51bbc07b09ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:32:02.278066 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987f208-b05f-44ce-9246-6e6a871edd20 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:02.334826 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:02.343030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 497/1011] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:32:02 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:32:02.355638 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:02.387914 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987f208-b05f-44ce-9246-6e6a871edd20 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:02.423196 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d2b36-cba4-4310-93bf-1d3e843f06c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:02.423850 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d2b36-cba4-4310-93bf-1d3e843f06c9 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:02.424131 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d2b36-cba4-4310-93bf-1d3e843f06c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:02.424401 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d2b36-cba4-4310-93bf-1d3e843f06c9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:02.425511 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6d2b36-cba4-4310-93bf-1d3e843f06c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:02.426914 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6d2b36-cba4-4310-93bf-1d3e843f06c9 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:02.429316 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 515/1012] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:32:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:32:02.434786 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803883f4-1d7f-4016-8e43-51bbc07b09ce None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:02.532771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803883f4-1d7f-4016-8e43-51bbc07b09ce None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:02.566969 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5fb27-fea1-4584-818e-82f4bb3f13a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:02.567601 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5fb27-fea1-4584-818e-82f4bb3f13a1 None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:02.567881 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5fb27-fea1-4584-818e-82f4bb3f13a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:02.568175 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5fb27-fea1-4584-818e-82f4bb3f13a1 None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:02.568846 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b5fb27-fea1-4584-818e-82f4bb3f13a1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:02.570249 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b5fb27-fea1-4584-818e-82f4bb3f13a1 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:02.619399 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d2b36-cba4-4310-93bf-1d3e843f06c9 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:02.667925 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 498/1013] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Oct 3 12:32:02 2022] POST /identity/v3/projects => generated 311 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:32:02.730607 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d2b36-cba4-4310-93bf-1d3e843f06c9 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:02.767541 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e79642-6273-4a49-8915-b78d11a1a322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:02.768186 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e79642-6273-4a49-8915-b78d11a1a322 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:02.768462 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e79642-6273-4a49-8915-b78d11a1a322 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:02.768729 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e79642-6273-4a49-8915-b78d11a1a322 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:02.769308 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e79642-6273-4a49-8915-b78d11a1a322 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:02.770769 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e79642-6273-4a49-8915-b78d11a1a322 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:02.780478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-c5e79642-6273-4a49-8915-b78d11a1a322 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 03 12:32:02.780478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:32:02.780478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 12:32:02.780478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 12:32:02.780478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 12:32:02.780478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 12:32:02.780478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 12:32:02.780478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:32:02.780478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 03 12:32:02.780478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:32:02.780478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 12:32:02.780478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:32:02.780478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:32:02.780478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:32:02.780478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:32:02.780478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:32:02.780478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:32:02.781587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:32:02.781587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:32:02.781587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:32:02.781587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:32:02.781587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:32:02.781587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:32:02.781587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 12:32:02.781587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 12:32:02.781587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 12:32:02.781587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 12:32:02.781587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:32:02.781587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:32:02.781587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:32:02.781587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:32:02.781587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:32:02.781587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:32:02.782461 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:32:02.782461 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:32:02.782461 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:32:02.782461 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:32:02.782461 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:32:02.782461 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:32:02.782461 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:32:02.782461 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:32:02.782461 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:32:02.782461 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:32:02.782461 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 12:32:02.782461 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:32:02.782461 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 03 12:32:02.782461 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:32:02.782461 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 516/1014] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:32:02 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:32:02.787754 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5fb27-fea1-4584-818e-82f4bb3f13a1 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:02.844151 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:02.898539 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5fb27-fea1-4584-818e-82f4bb3f13a1 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:02.933773 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56388620-89d8-464d-868a-43c21fb0debb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:02.934479 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56388620-89d8-464d-868a-43c21fb0debb None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:02.934694 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56388620-89d8-464d-868a-43c21fb0debb None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:02.934964 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56388620-89d8-464d-868a-43c21fb0debb None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:02.935629 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56388620-89d8-464d-868a-43c21fb0debb None admin] RBAC: Adding query filter params (name=compute) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:02.935927 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56388620-89d8-464d-868a-43c21fb0debb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:02.937345 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56388620-89d8-464d-868a-43c21fb0debb None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:03.000388 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 499/1015] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Oct 3 12:32:02 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:03.006788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e79642-6273-4a49-8915-b78d11a1a322 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:03.011437 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:03.118440 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e79642-6273-4a49-8915-b78d11a1a322 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:03.156370 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d7ae6-8708-4251-8016-febe754e21d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:03.156997 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d7ae6-8708-4251-8016-febe754e21d2 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:03.157277 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d7ae6-8708-4251-8016-febe754e21d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:03.157549 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d7ae6-8708-4251-8016-febe754e21d2 None admin] PATH_INFO: `/v3/services` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:03.158191 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8d7ae6-8708-4251-8016-febe754e21d2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:03.158494 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8d7ae6-8708-4251-8016-febe754e21d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:03.159987 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8d7ae6-8708-4251-8016-febe754e21d2 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:03.169980 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 517/1016] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Oct 3 12:32:03 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:03.171665 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56388620-89d8-464d-868a-43c21fb0debb None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:03.248686 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ab091-005f-4274-abb5-57354d28212d None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:03.248977 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ab091-005f-4274-abb5-57354d28212d None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:03.249252 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ab091-005f-4274-abb5-57354d28212d None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:03.250589 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 518/1017] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:32:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 12:32:03.256452 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b561433f-c760-46a3-9f5a-45ab5332bb47 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:03.256706 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b561433f-c760-46a3-9f5a-45ab5332bb47 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:03.256806 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b561433f-c760-46a3-9f5a-45ab5332bb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:03.275190 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-b561433f-c760-46a3-9f5a-45ab5332bb47 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:32:03.276417 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-b561433f-c760-46a3-9f5a-45ab5332bb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:32:03.292559 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56388620-89d8-464d-868a-43c21fb0debb None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:03.306557 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:03.332690 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28e8a26-2bce-4f3b-bc07-b594c8988263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:03.334509 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e8a26-2bce-4f3b-bc07-b594c8988263 None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:03.334509 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e8a26-2bce-4f3b-bc07-b594c8988263 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:03.334509 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e8a26-2bce-4f3b-bc07-b594c8988263 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:03.335172 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28e8a26-2bce-4f3b-bc07-b594c8988263 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:03.336581 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28e8a26-2bce-4f3b-bc07-b594c8988263 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:03.342083 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:03.347942 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 519/1018] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:32:03 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 12:32:03.427445 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 500/1019] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:32:03 2022] POST /identity/v3/endpoints => generated 334 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:32:03.628386 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b561433f-c760-46a3-9f5a-45ab5332bb47 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:03.735729 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b561433f-c760-46a3-9f5a-45ab5332bb47 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:03.777746 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd944d4-45e9-4223-92d1-424d7929818c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:03.778361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd944d4-45e9-4223-92d1-424d7929818c None admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:03.778648 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd944d4-45e9-4223-92d1-424d7929818c None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:03.778923 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd944d4-45e9-4223-92d1-424d7929818c None admin] PATH_INFO: `/v3/services` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:03.779598 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd944d4-45e9-4223-92d1-424d7929818c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:03.780950 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd944d4-45e9-4223-92d1-424d7929818c None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:03.893397 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 520/1020] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Oct 3 12:32:03 2022] POST /identity/v3/services => generated 242 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:32:04.906623 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb1ec1-5c85-4489-8b38-b70f7a163880 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:04.907152 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb1ec1-5c85-4489-8b38-b70f7a163880 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:04.907152 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb1ec1-5c85-4489-8b38-b70f7a163880 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:04.908662 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 501/1021] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:32:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:32:04.914390 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47292b26-c114-4151-9927-b74cc3a37d70 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:04.919738 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47292b26-c114-4151-9927-b74cc3a37d70 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:04.919738 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47292b26-c114-4151-9927-b74cc3a37d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:04.934178 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-47292b26-c114-4151-9927-b74cc3a37d70 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:32:04.935401 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-47292b26-c114-4151-9927-b74cc3a37d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:32:04.988632 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:05.015361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 521/1022] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:32:04 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:32:05.279925 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb1ec1-5c85-4489-8b38-b70f7a163880 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:05.384522 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb1ec1-5c85-4489-8b38-b70f7a163880 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:05.419925 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a46ddc-2555-4c64-9e94-4ee45b83fac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:05.420556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a46ddc-2555-4c64-9e94-4ee45b83fac5 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:05.420832 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a46ddc-2555-4c64-9e94-4ee45b83fac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:05.421108 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a46ddc-2555-4c64-9e94-4ee45b83fac5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:05.422202 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a46ddc-2555-4c64-9e94-4ee45b83fac5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:05.423635 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a46ddc-2555-4c64-9e94-4ee45b83fac5 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:05.425346 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 502/1023] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:32:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:05.431269 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47292b26-c114-4151-9927-b74cc3a37d70 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:05.537135 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47292b26-c114-4151-9927-b74cc3a37d70 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:05.573346 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25b41a-7d9b-4ea5-b447-21374e0fab07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:05.573991 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25b41a-7d9b-4ea5-b447-21374e0fab07 None admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:05.574276 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25b41a-7d9b-4ea5-b447-21374e0fab07 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:05.574552 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25b41a-7d9b-4ea5-b447-21374e0fab07 None admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:05.575237 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac25b41a-7d9b-4ea5-b447-21374e0fab07 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:05.576757 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac25b41a-7d9b-4ea5-b447-21374e0fab07 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:05.692490 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00cea9d-a49d-4464-b72b-ca1af7b8eb5e None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:05.692857 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00cea9d-a49d-4464-b72b-ca1af7b8eb5e None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:05.693039 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00cea9d-a49d-4464-b72b-ca1af7b8eb5e None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:05.699674 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 522/1024] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Oct 3 12:32:05 2022] POST /identity/v3/users => generated 285 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:32:05.702460 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 503/1025] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:32:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 12:32:05.702591 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bdeb9b-8ba0-43d8-bcc0-db4cd165faec None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:05.702591 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bdeb9b-8ba0-43d8-bcc0-db4cd165faec None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:05.702591 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bdeb9b-8ba0-43d8-bcc0-db4cd165faec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:05.723618 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-38bdeb9b-8ba0-43d8-bcc0-db4cd165faec None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:32:05.726303 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-38bdeb9b-8ba0-43d8-bcc0-db4cd165faec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:32:05.760879 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:05.805939 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 523/1026] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:32:05 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:32:06.074145 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00cea9d-a49d-4464-b72b-ca1af7b8eb5e None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:06.094248 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021f5bb-5f9c-42a2-a601-62ff79825bad None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:06.094753 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021f5bb-5f9c-42a2-a601-62ff79825bad None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:06.095031 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021f5bb-5f9c-42a2-a601-62ff79825bad None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:06.096756 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 524/1027] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:32:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:32:06.102266 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661ab53-9d5f-4c41-b019-2c4f77fdce3d None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:06.102665 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661ab53-9d5f-4c41-b019-2c4f77fdce3d None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:06.102937 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661ab53-9d5f-4c41-b019-2c4f77fdce3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:06.114198 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:06.121619 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-3661ab53-9d5f-4c41-b019-2c4f77fdce3d None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:32:06.122798 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-3661ab53-9d5f-4c41-b019-2c4f77fdce3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:32:06.187767 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00cea9d-a49d-4464-b72b-ca1af7b8eb5e None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:06.198397 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 525/1028] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:32:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:32:06.225812 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d507f-ee1c-45bb-abca-8373b75d1d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:06.226453 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d507f-ee1c-45bb-abca-8373b75d1d0c None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:06.226731 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d507f-ee1c-45bb-abca-8373b75d1d0c None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:06.227017 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d507f-ee1c-45bb-abca-8373b75d1d0c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:06.243419 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6d507f-ee1c-45bb-abca-8373b75d1d0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:32:06.249758 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-fe6d507f-ee1c-45bb-abca-8373b75d1d0c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 12:32:06.249758 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:32:06.249758 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:32:06.249758 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:32:06.249758 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:32:06.249758 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:32:06.249758 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:32:06.249758 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:32:06.249758 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:32:06.249758 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:32:06.249758 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:32:06.249758 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:32:06.249758 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:32:06.249758 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:32:06.249758 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:32:06.249758 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:32:06.250651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:32:06.250651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:32:06.250651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:32:06.250651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:32:06.250651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:32:06.250651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:32:06.250651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:32:06.250651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:32:06.250651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:32:06.250651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:32:06.250651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:32:06.250651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:32:06.250651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:32:06.250651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:32:06.250651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:32:06.250651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:32:06.251590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:32:06.251590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:32:06.251590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:32:06.251590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:32:06.251590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:32:06.251590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:32:06.251590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:32:06.251590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:32:06.251590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:32:06.251590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:32:06.251590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 12:32:06.251590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:32:06.251590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 504/1029] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Oct 3 12:32:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:32:06.257420 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3661ab53-9d5f-4c41-b019-2c4f77fdce3d None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:06.358358 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:06.364028 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3661ab53-9d5f-4c41-b019-2c4f77fdce3d None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:06.398527 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f429df-389d-4c5e-b1bd-c8ff6f583829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:06.399136 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f429df-389d-4c5e-b1bd-c8ff6f583829 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:06.399464 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f429df-389d-4c5e-b1bd-c8ff6f583829 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:06.399737 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f429df-389d-4c5e-b1bd-c8ff6f583829 None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:06.400397 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f429df-389d-4c5e-b1bd-c8ff6f583829 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:06.400697 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f429df-389d-4c5e-b1bd-c8ff6f583829 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:06.402042 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f429df-389d-4c5e-b1bd-c8ff6f583829 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:06.416096 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 526/1030] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Oct 3 12:32:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:06.417737 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d507f-ee1c-45bb-abca-8373b75d1d0c None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:06.471187 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f429df-389d-4c5e-b1bd-c8ff6f583829 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:06.516319 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:06.522679 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d507f-ee1c-45bb-abca-8373b75d1d0c None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:06.559328 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bac568d-4f01-4ea0-846f-57636c25ec09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:06.559980 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac568d-4f01-4ea0-846f-57636c25ec09 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:06.560368 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac568d-4f01-4ea0-846f-57636c25ec09 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:06.560641 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac568d-4f01-4ea0-846f-57636c25ec09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:06.572136 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bac568d-4f01-4ea0-846f-57636c25ec09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:06.573817 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bac568d-4f01-4ea0-846f-57636c25ec09 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:06.581683 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f429df-389d-4c5e-b1bd-c8ff6f583829 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:06.589958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-9bac568d-4f01-4ea0-846f-57636c25ec09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:32:06.589958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:32:06.589958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:32:06.589958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:32:06.589958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:32:06.589958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:32:06.589958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:32:06.589958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:32:06.589958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:32:06.589958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:32:06.589958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:32:06.589958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:32:06.589958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:32:06.589958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:32:06.589958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:32:06.589958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:32:06.590926 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:32:06.590926 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:32:06.590926 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:32:06.590926 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:32:06.590926 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:32:06.590926 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:32:06.590926 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:32:06.590926 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:32:06.590926 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:32:06.590926 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:32:06.590926 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:32:06.590926 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:32:06.590926 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:32:06.590926 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:32:06.590926 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:32:06.590926 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:32:06.590926 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:32:06.592528 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:32:06.592528 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:32:06.592528 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:32:06.592528 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 505/1031] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:32:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:32:06.596975 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bac568d-4f01-4ea0-846f-57636c25ec09 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:06.622484 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95638a58-f5b9-4e86-82b3-1b1df40699da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:06.623051 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95638a58-f5b9-4e86-82b3-1b1df40699da None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:06.623354 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95638a58-f5b9-4e86-82b3-1b1df40699da None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:06.623632 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95638a58-f5b9-4e86-82b3-1b1df40699da None admin] PATH_INFO: `/v3/services/image` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:06.624227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95638a58-f5b9-4e86-82b3-1b1df40699da None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:06.625552 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95638a58-f5b9-4e86-82b3-1b1df40699da None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:06.634636 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-95638a58-f5b9-4e86-82b3-1b1df40699da None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 03 12:32:06.634636 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:32:06.634636 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 12:32:06.634636 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 12:32:06.634636 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 12:32:06.634636 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 12:32:06.634636 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 12:32:06.634636 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:32:06.634636 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 03 12:32:06.634636 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:32:06.634636 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 12:32:06.634636 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:32:06.634636 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:32:06.634636 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:32:06.634636 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:32:06.634636 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:32:06.634636 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:32:06.635740 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:32:06.635740 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:32:06.635740 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:32:06.635740 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:32:06.635740 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:32:06.635740 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:32:06.635740 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 12:32:06.635740 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 12:32:06.635740 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 12:32:06.635740 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 12:32:06.635740 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:32:06.635740 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:32:06.635740 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:32:06.635740 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:32:06.635740 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:32:06.635740 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:32:06.636586 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:32:06.636586 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:32:06.636586 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:32:06.636586 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:32:06.636586 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:32:06.636586 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:32:06.636586 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:32:06.636586 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:32:06.636586 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:32:06.636586 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:32:06.636586 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 12:32:06.636586 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:32:06.636586 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 03 12:32:06.636586 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:32:06.636586 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 527/1032] 10.222.0.33 () {56 vars in 1105 bytes} [Mon Oct 3 12:32:06 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:32:06.642214 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95638a58-f5b9-4e86-82b3-1b1df40699da None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:06.669934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:06.711422 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bac568d-4f01-4ea0-846f-57636c25ec09 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:06.749244 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea85f0-e896-4a7b-8778-2d3fd42ce374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:06.749859 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea85f0-e896-4a7b-8778-2d3fd42ce374 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:06.750142 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea85f0-e896-4a7b-8778-2d3fd42ce374 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:06.750417 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea85f0-e896-4a7b-8778-2d3fd42ce374 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:06.751051 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfea85f0-e896-4a7b-8778-2d3fd42ce374 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:06.751376 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfea85f0-e896-4a7b-8778-2d3fd42ce374 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:06.752741 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfea85f0-e896-4a7b-8778-2d3fd42ce374 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:06.753007 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95638a58-f5b9-4e86-82b3-1b1df40699da None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:06.765252 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 506/1033] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:32:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:32:06.770768 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea85f0-e896-4a7b-8778-2d3fd42ce374 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:06.781972 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:06.795950 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0f46e-625a-41f3-a4bf-9f7da14e9386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:06.796886 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0f46e-625a-41f3-a4bf-9f7da14e9386 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:06.811684 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0f46e-625a-41f3-a4bf-9f7da14e9386 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:06.811684 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0f46e-625a-41f3-a4bf-9f7da14e9386 None admin] PATH_INFO: `/v3/services` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:06.811684 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f0f46e-625a-41f3-a4bf-9f7da14e9386 None admin] RBAC: Adding query filter params (name=image) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:06.811684 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f0f46e-625a-41f3-a4bf-9f7da14e9386 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:06.811684 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f0f46e-625a-41f3-a4bf-9f7da14e9386 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:06.811684 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 528/1034] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:32:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:32:06.812997 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0f46e-625a-41f3-a4bf-9f7da14e9386 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:06.874832 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea85f0-e896-4a7b-8778-2d3fd42ce374 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:06.906629 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:06.911290 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab621dd0-bd55-4bd7-80df-e3cc2d0744be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:06.912487 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab621dd0-bd55-4bd7-80df-e3cc2d0744be None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:06.914814 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab621dd0-bd55-4bd7-80df-e3cc2d0744be None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:06.914814 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab621dd0-bd55-4bd7-80df-e3cc2d0744be None admin] PATH_INFO: `/v3/users/nova` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:06.921008 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab621dd0-bd55-4bd7-80df-e3cc2d0744be None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:06.922574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab621dd0-bd55-4bd7-80df-e3cc2d0744be None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:06.922959 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0f46e-625a-41f3-a4bf-9f7da14e9386 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:06.936773 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-ab621dd0-bd55-4bd7-80df-e3cc2d0744be None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 03 12:32:06.936773 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:32:06.936773 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:32:06.936773 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:32:06.936773 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:32:06.936773 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:32:06.936773 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:32:06.936773 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:32:06.936773 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:32:06.936773 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:32:06.936773 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:32:06.936773 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:32:06.936773 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 12:32:06.936773 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 12:32:06.936773 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 12:32:06.936773 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 12:32:06.943048 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:32:06.943048 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:32:06.943048 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 12:32:06.943048 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:32:06.943048 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 12:32:06.943048 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:32:06.943048 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:32:06.943048 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:32:06.943048 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:32:06.943048 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:32:06.943048 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:32:06.943048 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:32:06.943048 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:32:06.943048 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:32:06.943048 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:32:06.943048 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:32:06.943048 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:32:06.943048 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:32:06.944336 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:32:06.944336 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:32:06.944336 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 12:32:06.944336 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 12:32:06.944336 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 12:32:06.944336 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 12:32:06.944336 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 12:32:06.944336 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 12:32:06.944336 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 03 12:32:06.944336 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:32:06.944336 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 507/1035] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Oct 3 12:32:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:32:06.944336 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab621dd0-bd55-4bd7-80df-e3cc2d0744be None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:06.959646 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83854383-4328-403b-b91d-0627c7d18b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:06.960281 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83854383-4328-403b-b91d-0627c7d18b0b None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:06.960572 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83854383-4328-403b-b91d-0627c7d18b0b None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:06.961229 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83854383-4328-403b-b91d-0627c7d18b0b None admin] PATH_INFO: `/v3/services` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:06.961865 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83854383-4328-403b-b91d-0627c7d18b0b None admin] RBAC: Adding query filter params (type=image) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:06.962525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83854383-4328-403b-b91d-0627c7d18b0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:06.964328 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83854383-4328-403b-b91d-0627c7d18b0b None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:06.969695 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 529/1036] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:32:06 2022] GET /identity/v3/services?type=image => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:06.976995 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83854383-4328-403b-b91d-0627c7d18b0b None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:07.017435 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:07.054649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab621dd0-bd55-4bd7-80df-e3cc2d0744be None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:07.078920 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83854383-4328-403b-b91d-0627c7d18b0b None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:07.093929 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171eadc-ae88-4552-b337-ec988fe9b708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:07.094853 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171eadc-ae88-4552-b337-ec988fe9b708 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:07.095168 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171eadc-ae88-4552-b337-ec988fe9b708 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:07.095489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171eadc-ae88-4552-b337-ec988fe9b708 None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:07.096228 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a171eadc-ae88-4552-b337-ec988fe9b708 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:07.096530 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a171eadc-ae88-4552-b337-ec988fe9b708 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:07.097971 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a171eadc-ae88-4552-b337-ec988fe9b708 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:07.115764 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf206c0-4263-46a4-9b57-967a0e2b59c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:07.120848 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 508/1037] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Oct 3 12:32:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:32:07.120956 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf206c0-4263-46a4-9b57-967a0e2b59c8 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:07.120956 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf206c0-4263-46a4-9b57-967a0e2b59c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:07.120956 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf206c0-4263-46a4-9b57-967a0e2b59c8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:07.120956 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf206c0-4263-46a4-9b57-967a0e2b59c8 None admin] RBAC: Adding query filter params (interface=public, service_id=966f8381cabb473198eeda937191a551, region_id=RegionOne) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:07.120956 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf206c0-4263-46a4-9b57-967a0e2b59c8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:07.120956 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf206c0-4263-46a4-9b57-967a0e2b59c8 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:07.121659 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171eadc-ae88-4552-b337-ec988fe9b708 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:07.125627 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 530/1038] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Oct 3 12:32:06 2022] GET /identity/v3/endpoints?service_id=966f8381cabb473198eeda937191a551&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:07.131120 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf206c0-4263-46a4-9b57-967a0e2b59c8 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:07.148080 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:07.230136 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171eadc-ae88-4552-b337-ec988fe9b708 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:07.242448 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf206c0-4263-46a4-9b57-967a0e2b59c8 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:07.258108 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:07.266789 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f64b2-da72-4442-90e7-ea8c6dde952f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:07.267181 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f64b2-da72-4442-90e7-ea8c6dde952f None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:07.268004 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f64b2-da72-4442-90e7-ea8c6dde952f None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:07.268004 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f64b2-da72-4442-90e7-ea8c6dde952f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:07.282026 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8525a5-2284-4d45-afe2-e447da7b3dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:07.282736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8525a5-2284-4d45-afe2-e447da7b3dd2 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:07.283094 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8525a5-2284-4d45-afe2-e447da7b3dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:07.283481 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8525a5-2284-4d45-afe2-e447da7b3dd2 None admin] PATH_INFO: `/v3/services` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:07.284157 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8525a5-2284-4d45-afe2-e447da7b3dd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:07.285598 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8525a5-2284-4d45-afe2-e447da7b3dd2 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:07.292746 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 531/1039] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Oct 3 12:32:07 2022] GET /identity/v3/services => generated 1957 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 12:32:07.347517 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316f64b2-da72-4442-90e7-ea8c6dde952f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:07.349007 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316f64b2-da72-4442-90e7-ea8c6dde952f None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:07.465175 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 509/1040] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:32:07 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 03 12:32:07.466544 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8525a5-2284-4d45-afe2-e447da7b3dd2 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:07.552867 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:07.575523 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8525a5-2284-4d45-afe2-e447da7b3dd2 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:07.612054 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f70f2e-cf28-42b1-a5cb-b7f1664f7b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:07.613060 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f70f2e-cf28-42b1-a5cb-b7f1664f7b3b None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:07.613339 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f70f2e-cf28-42b1-a5cb-b7f1664f7b3b None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:07.613604 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f70f2e-cf28-42b1-a5cb-b7f1664f7b3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:07.623976 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f70f2e-cf28-42b1-a5cb-b7f1664f7b3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:07.625402 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f70f2e-cf28-42b1-a5cb-b7f1664f7b3b None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:07.640563 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-f8f70f2e-cf28-42b1-a5cb-b7f1664f7b3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:32:07.640563 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:32:07.640563 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:32:07.640563 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:32:07.640563 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:32:07.640563 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:32:07.640563 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:32:07.640563 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:32:07.640563 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:32:07.640563 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:32:07.640563 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:32:07.640563 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:32:07.640563 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:32:07.640563 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:32:07.640563 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:32:07.640563 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:32:07.641502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:32:07.641502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:32:07.641502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:32:07.641502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:32:07.641502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:32:07.641502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:32:07.641502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:32:07.641502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:32:07.641502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:32:07.641502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:32:07.641502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:32:07.641502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:32:07.641502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:32:07.641502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:32:07.641502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:32:07.641502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:32:07.641502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:32:07.642469 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:32:07.642469 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:32:07.642469 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:32:07.642469 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 532/1041] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:32:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:32:07.648901 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f64b2-da72-4442-90e7-ea8c6dde952f None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:07.676238 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:07.750406 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f64b2-da72-4442-90e7-ea8c6dde952f None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:07.785433 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b5e86e-5e77-407d-ae5c-97c92ec3b89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:07.786052 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5e86e-5e77-407d-ae5c-97c92ec3b89f None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:07.788582 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5e86e-5e77-407d-ae5c-97c92ec3b89f None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:07.788582 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5e86e-5e77-407d-ae5c-97c92ec3b89f None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:07.788582 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b5e86e-5e77-407d-ae5c-97c92ec3b89f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:07.788582 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b5e86e-5e77-407d-ae5c-97c92ec3b89f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:07.789595 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b5e86e-5e77-407d-ae5c-97c92ec3b89f None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:07.803501 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 510/1042] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:32:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:07.807010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f70f2e-cf28-42b1-a5cb-b7f1664f7b3b None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:07.912748 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f70f2e-cf28-42b1-a5cb-b7f1664f7b3b None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:07.933466 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27730109-4722-4f89-ba36-5eb706c245e0 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:07.933748 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27730109-4722-4f89-ba36-5eb706c245e0 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:07.934034 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27730109-4722-4f89-ba36-5eb706c245e0 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:07.935676 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 511/1043] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:32:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 12:32:07.941407 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79341c9d-6f7a-44fd-9298-42488c05e50e None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:07.941712 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79341c9d-6f7a-44fd-9298-42488c05e50e None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:07.941929 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79341c9d-6f7a-44fd-9298-42488c05e50e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:07.946954 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:07.953793 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11622cae-e6d4-48a2-adb6-9ea0ff162a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:07.954431 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11622cae-e6d4-48a2-adb6-9ea0ff162a72 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:07.954711 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11622cae-e6d4-48a2-adb6-9ea0ff162a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:07.954983 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11622cae-e6d4-48a2-adb6-9ea0ff162a72 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:07.961776 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-79341c9d-6f7a-44fd-9298-42488c05e50e None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:32:07.962816 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11622cae-e6d4-48a2-adb6-9ea0ff162a72 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:07.963437 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-79341c9d-6f7a-44fd-9298-42488c05e50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:32:07.964277 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11622cae-e6d4-48a2-adb6-9ea0ff162a72 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:07.977758 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-11622cae-e6d4-48a2-adb6-9ea0ff162a72 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:32:07.977758 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:32:07.977758 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:32:07.977758 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:32:07.977758 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:32:07.977758 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:32:07.977758 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:32:07.977758 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:32:07.977758 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:32:07.977758 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:32:07.977758 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:32:07.977758 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:32:07.977758 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 12:32:07.977758 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 12:32:07.977758 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 12:32:07.978693 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 12:32:07.978693 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:32:07.978693 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:32:07.978693 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:32:07.978693 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:32:07.978693 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:32:07.978693 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:32:07.978693 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:32:07.978693 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:32:07.978693 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:32:07.978693 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:32:07.978693 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:32:07.978693 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:32:07.978693 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:32:07.978693 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:32:07.978693 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:32:07.978693 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:32:07.979607 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 12:32:07.979607 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 12:32:07.979607 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 12:32:07.979607 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 12:32:07.979607 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 12:32:07.979607 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 12:32:07.979607 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:32:07.979607 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:32:07.979607 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 533/1044] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:32:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:32:07.984706 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11622cae-e6d4-48a2-adb6-9ea0ff162a72 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:08.048058 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 512/1045] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:32:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 12:32:08.091742 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11622cae-e6d4-48a2-adb6-9ea0ff162a72 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:08.127228 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f50483f-d59b-49d2-9473-7d96ae89036c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:08.127872 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50483f-d59b-49d2-9473-7d96ae89036c None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:08.128134 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50483f-d59b-49d2-9473-7d96ae89036c None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:08.128406 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50483f-d59b-49d2-9473-7d96ae89036c None admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:08.129094 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f50483f-d59b-49d2-9473-7d96ae89036c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:08.129401 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f50483f-d59b-49d2-9473-7d96ae89036c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:08.131617 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f50483f-d59b-49d2-9473-7d96ae89036c None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:08.149213 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 534/1046] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 12:32:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:32:08.150746 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79341c9d-6f7a-44fd-9298-42488c05e50e None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:08.172239 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:08.264880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79341c9d-6f7a-44fd-9298-42488c05e50e None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:08.302688 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c4bff9-5068-4738-8693-85818b4b0763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:08.303923 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4bff9-5068-4738-8693-85818b4b0763 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:08.304197 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4bff9-5068-4738-8693-85818b4b0763 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:08.304465 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4bff9-5068-4738-8693-85818b4b0763 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:08.305181 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c4bff9-5068-4738-8693-85818b4b0763 None admin] RBAC: Adding query filter params (role.id=a7c026bed2224838b2a71641eb61b9d0, scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7, user.id=990f5837994f4a088c656c9b33e9a0dc) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:08.305483 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c4bff9-5068-4738-8693-85818b4b0763 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:08.306894 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c4bff9-5068-4738-8693-85818b4b0763 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:08.311907 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f50483f-d59b-49d2-9473-7d96ae89036c None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:08.312812 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 513/1047] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:32:08 2022] GET /identity/v3/role_assignments?user.id=990f5837994f4a088c656c9b33e9a0dc&scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7&role.id=a7c026bed2224838b2a71641eb61b9d0 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:08.371364 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:08.426280 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f50483f-d59b-49d2-9473-7d96ae89036c None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:08.472421 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735e02e-abcd-4af5-9d0b-5a7c90c92c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:08.473068 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735e02e-abcd-4af5-9d0b-5a7c90c92c9b None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:08.473347 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735e02e-abcd-4af5-9d0b-5a7c90c92c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:08.473616 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735e02e-abcd-4af5-9d0b-5a7c90c92c9b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:08.485029 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9735e02e-abcd-4af5-9d0b-5a7c90c92c9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:32:08.490363 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-9735e02e-abcd-4af5-9d0b-5a7c90c92c9b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:32:08.490363 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:32:08.490363 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:32:08.490363 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:32:08.490363 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:32:08.490363 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:32:08.490363 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:32:08.490363 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:32:08.490363 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:32:08.490363 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:32:08.490363 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:32:08.490363 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:32:08.490363 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:32:08.490363 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:32:08.490363 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:32:08.490363 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:32:08.495629 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:32:08.495629 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:32:08.495629 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:32:08.495629 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:32:08.495629 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:32:08.495629 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:32:08.495629 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:32:08.495629 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:32:08.495629 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:32:08.495629 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:32:08.495629 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:32:08.495629 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:32:08.495629 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:32:08.495629 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:32:08.495629 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:32:08.495629 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:32:08.496485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:32:08.496485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:32:08.496485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:32:08.496485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:32:08.496485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:32:08.496485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:32:08.496485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:32:08.496485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:32:08.496485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:32:08.496485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:32:08.496485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:32:08.496485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:32:08.496485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 535/1048] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 12:32:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:32:08.497938 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c4bff9-5068-4738-8693-85818b4b0763 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:08.585558 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:08.604743 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c4bff9-5068-4738-8693-85818b4b0763 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:08.641711 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3fb9f4-78aa-4079-860e-d4c7330d2bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:08.642329 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3fb9f4-78aa-4079-860e-d4c7330d2bdb None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:08.642608 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3fb9f4-78aa-4079-860e-d4c7330d2bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:08.642881 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3fb9f4-78aa-4079-860e-d4c7330d2bdb None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:08.643568 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3fb9f4-78aa-4079-860e-d4c7330d2bdb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:08.643873 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3fb9f4-78aa-4079-860e-d4c7330d2bdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:08.645230 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3fb9f4-78aa-4079-860e-d4c7330d2bdb None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:08.654848 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 514/1049] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:32:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:08.660541 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735e02e-abcd-4af5-9d0b-5a7c90c92c9b None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:08.766736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735e02e-abcd-4af5-9d0b-5a7c90c92c9b None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:08.778566 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:08.803415 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f51be-0134-47c3-8811-4069d394bb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:08.804426 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f51be-0134-47c3-8811-4069d394bb3b None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:08.804708 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f51be-0134-47c3-8811-4069d394bb3b None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:08.804982 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f51be-0134-47c3-8811-4069d394bb3b None admin] PATH_INFO: `/v3/users/e6c7b23c70cc49a0a106e38e514ac54c` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:08.822264 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5f51be-0134-47c3-8811-4069d394bb3b None admin] RBAC: Authorizing `identity:get_user(user_id=e6c7b23c70cc49a0a106e38e514ac54c)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:08.823747 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5f51be-0134-47c3-8811-4069d394bb3b None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:08.830110 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 536/1050] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:32:08 2022] GET /identity/v3/users/e6c7b23c70cc49a0a106e38e514ac54c => generated 285 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:08.831165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3fb9f4-78aa-4079-860e-d4c7330d2bdb None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:08.932548 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3fb9f4-78aa-4079-860e-d4c7330d2bdb None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:08.966892 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cdcba1-5a3c-45a4-ad02-70594f6fd8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:08.967525 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cdcba1-5a3c-45a4-ad02-70594f6fd8a6 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:08.967802 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cdcba1-5a3c-45a4-ad02-70594f6fd8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:08.968068 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cdcba1-5a3c-45a4-ad02-70594f6fd8a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:08.981599 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:09.036632 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cdcba1-5a3c-45a4-ad02-70594f6fd8a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:09.038237 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cdcba1-5a3c-45a4-ad02-70594f6fd8a6 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:09.146072 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 515/1051] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:32:08 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 03 12:32:09.147383 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f51be-0134-47c3-8811-4069d394bb3b None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:09.258379 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f51be-0134-47c3-8811-4069d394bb3b None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:09.292221 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb078957-9537-4729-ab7f-5e9cd8ebfb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:09.292845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb078957-9537-4729-ab7f-5e9cd8ebfb3e None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:09.293123 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb078957-9537-4729-ab7f-5e9cd8ebfb3e None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:09.293392 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb078957-9537-4729-ab7f-5e9cd8ebfb3e None admin] PATH_INFO: `/v3/projects/c73bdc87381d4b479b5c19adf5d3c06d` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:09.294481 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb078957-9537-4729-ab7f-5e9cd8ebfb3e None admin] RBAC: Authorizing `identity:get_project(project_id=c73bdc87381d4b479b5c19adf5d3c06d)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:09.295913 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb078957-9537-4729-ab7f-5e9cd8ebfb3e None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:09.302479 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 537/1052] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:32:09 2022] GET /identity/v3/projects/c73bdc87381d4b479b5c19adf5d3c06d => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:09.304527 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cdcba1-5a3c-45a4-ad02-70594f6fd8a6 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:09.355522 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:09.433886 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cdcba1-5a3c-45a4-ad02-70594f6fd8a6 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:09.470889 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e4840-0afc-44a1-ab0a-f7aa3a965511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:09.471529 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e4840-0afc-44a1-ab0a-f7aa3a965511 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:09.471804 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e4840-0afc-44a1-ab0a-f7aa3a965511 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:09.472068 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e4840-0afc-44a1-ab0a-f7aa3a965511 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:09.472780 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e4840-0afc-44a1-ab0a-f7aa3a965511 None admin] RBAC: Adding query filter params (role.id=7933c45dcdd54ef0a98018ba8a8bddaa, scope.project.id=c73bdc87381d4b479b5c19adf5d3c06d, user.id=e6c7b23c70cc49a0a106e38e514ac54c) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:09.473077 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e4840-0afc-44a1-ab0a-f7aa3a965511 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:09.475181 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e4840-0afc-44a1-ab0a-f7aa3a965511 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:09.487580 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 516/1053] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:32:09 2022] GET /identity/v3/role_assignments?user.id=e6c7b23c70cc49a0a106e38e514ac54c&scope.project.id=c73bdc87381d4b479b5c19adf5d3c06d&role.id=7933c45dcdd54ef0a98018ba8a8bddaa => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:09.504207 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1187cb0e-94af-42df-ba67-d2e39e537506 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:09.504457 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1187cb0e-94af-42df-ba67-d2e39e537506 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:09.504681 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1187cb0e-94af-42df-ba67-d2e39e537506 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:09.506104 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 538/1054] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:32:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:32:09.512089 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd103b-91c8-4191-a15c-8aeb463e10a9 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:09.512397 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd103b-91c8-4191-a15c-8aeb463e10a9 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:09.512639 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd103b-91c8-4191-a15c-8aeb463e10a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:09.531994 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-61fd103b-91c8-4191-a15c-8aeb463e10a9 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:32:09.533230 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-61fd103b-91c8-4191-a15c-8aeb463e10a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:32:09.608465 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 517/1055] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:32:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:32:09.876094 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1187cb0e-94af-42df-ba67-d2e39e537506 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:09.914113 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:09.981460 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1187cb0e-94af-42df-ba67-d2e39e537506 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:10.015798 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02674f43-1826-4d42-9020-956bb9f448ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:10.016427 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02674f43-1826-4d42-9020-956bb9f448ed None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:10.016703 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02674f43-1826-4d42-9020-956bb9f448ed None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:10.016975 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02674f43-1826-4d42-9020-956bb9f448ed None admin] PATH_INFO: `/v3/services/image` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:10.017543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02674f43-1826-4d42-9020-956bb9f448ed None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:10.018872 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02674f43-1826-4d42-9020-956bb9f448ed None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:10.028086 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-02674f43-1826-4d42-9020-956bb9f448ed None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 03 12:32:10.028086 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:32:10.028086 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 12:32:10.028086 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 12:32:10.028086 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 12:32:10.028086 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 12:32:10.028086 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 12:32:10.028086 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:32:10.028086 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 03 12:32:10.028086 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:32:10.028086 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 12:32:10.028086 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:32:10.028086 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:32:10.028086 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:32:10.028086 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:32:10.028086 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:32:10.028086 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:32:10.029115 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:32:10.029115 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:32:10.029115 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:32:10.029115 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:32:10.029115 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:32:10.029115 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:32:10.029115 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 12:32:10.029115 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 12:32:10.029115 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 12:32:10.029115 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 12:32:10.029115 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:32:10.029115 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:32:10.029115 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:32:10.029115 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:32:10.029115 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:32:10.029115 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:32:10.029986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:32:10.029986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:32:10.029986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:32:10.029986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:32:10.029986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:32:10.029986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:32:10.029986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:32:10.029986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:32:10.029986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:32:10.029986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:32:10.029986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 12:32:10.029986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:32:10.029986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 03 12:32:10.029986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:32:10.031651 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 539/1056] 10.222.0.33 () {56 vars in 1105 bytes} [Mon Oct 3 12:32:09 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:32:10.034999 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fd103b-91c8-4191-a15c-8aeb463e10a9 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:10.068893 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:10.134294 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fd103b-91c8-4191-a15c-8aeb463e10a9 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:10.169049 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c97db-d4c1-489e-a5e3-837c756953c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:10.169663 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c97db-d4c1-489e-a5e3-837c756953c9 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:10.169938 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c97db-d4c1-489e-a5e3-837c756953c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:10.170205 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c97db-d4c1-489e-a5e3-837c756953c9 None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:10.170820 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0c97db-d4c1-489e-a5e3-837c756953c9 None admin] RBAC: Adding query filter params (name=image) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:10.171113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0c97db-d4c1-489e-a5e3-837c756953c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:10.172492 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0c97db-d4c1-489e-a5e3-837c756953c9 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:10.177673 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 518/1057] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:32:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:10.183234 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02674f43-1826-4d42-9020-956bb9f448ed None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:10.287789 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:10.290799 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02674f43-1826-4d42-9020-956bb9f448ed None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:10.326193 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f245ea0-c969-4a13-b1bb-6afdaecfb64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:10.326806 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f245ea0-c969-4a13-b1bb-6afdaecfb64a None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:10.327086 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f245ea0-c969-4a13-b1bb-6afdaecfb64a None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:10.327382 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f245ea0-c969-4a13-b1bb-6afdaecfb64a None admin] PATH_INFO: `/v3/services` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:10.328012 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f245ea0-c969-4a13-b1bb-6afdaecfb64a None admin] RBAC: Adding query filter params (type=image) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:10.328316 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f245ea0-c969-4a13-b1bb-6afdaecfb64a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:10.329699 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f245ea0-c969-4a13-b1bb-6afdaecfb64a None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:10.339609 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 540/1058] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:32:10 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:10.340432 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c97db-d4c1-489e-a5e3-837c756953c9 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:10.445494 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:10.448993 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c97db-d4c1-489e-a5e3-837c756953c9 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:10.483342 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823cf0da-7968-4206-8f19-d38bc4be80b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:10.483958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823cf0da-7968-4206-8f19-d38bc4be80b3 None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:10.484235 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823cf0da-7968-4206-8f19-d38bc4be80b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:10.484504 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823cf0da-7968-4206-8f19-d38bc4be80b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:10.485161 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823cf0da-7968-4206-8f19-d38bc4be80b3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:10.486500 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823cf0da-7968-4206-8f19-d38bc4be80b3 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:10.505184 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56251e4d-abf8-43ca-a348-d0aad32d3522 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:10.505466 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56251e4d-abf8-43ca-a348-d0aad32d3522 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:10.506278 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56251e4d-abf8-43ca-a348-d0aad32d3522 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:10.507825 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 541/1059] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:32:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 12:32:10.512534 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a61398-550b-4440-b2c7-fb3305d8c2bc None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:10.512821 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a61398-550b-4440-b2c7-fb3305d8c2bc None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:10.513097 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a61398-550b-4440-b2c7-fb3305d8c2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:10.531260 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-f0a61398-550b-4440-b2c7-fb3305d8c2bc None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:32:10.532663 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-f0a61398-550b-4440-b2c7-fb3305d8c2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:32:10.558773 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 519/1060] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:32:10 2022] POST /identity/v3/endpoints => generated 327 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:32:10.623677 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 542/1061] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:32:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 12:32:10.890908 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823cf0da-7968-4206-8f19-d38bc4be80b3 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:10.995488 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823cf0da-7968-4206-8f19-d38bc4be80b3 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:11.030084 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d020b-b085-4111-88fb-2dcd173e4c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:11.030734 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d020b-b085-4111-88fb-2dcd173e4c9b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:11.031014 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d020b-b085-4111-88fb-2dcd173e4c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:11.031288 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d020b-b085-4111-88fb-2dcd173e4c9b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:11.042674 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d020b-b085-4111-88fb-2dcd173e4c9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:32:11.047496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-a15d020b-b085-4111-88fb-2dcd173e4c9b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 12:32:11.047496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:32:11.047496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:32:11.047496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:32:11.047496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:32:11.047496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:32:11.047496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:32:11.047496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:32:11.047496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:32:11.047496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:32:11.047496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:32:11.047496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:32:11.047496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:32:11.047496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:32:11.047496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:32:11.047496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:32:11.048396 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:32:11.048396 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:32:11.048396 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:32:11.048396 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:32:11.048396 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:32:11.048396 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:32:11.048396 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:32:11.048396 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:32:11.048396 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:32:11.048396 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:32:11.048396 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:32:11.048396 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:32:11.048396 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:32:11.048396 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:32:11.048396 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:32:11.048396 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:32:11.049847 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:32:11.049847 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:32:11.049847 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:32:11.049847 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:32:11.049847 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:32:11.049847 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:32:11.049847 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:32:11.049847 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:32:11.049847 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:32:11.049847 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:32:11.049847 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 12:32:11.049847 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:32:11.049847 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 520/1062] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Oct 3 12:32:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:32:11.055300 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a61398-550b-4440-b2c7-fb3305d8c2bc None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:11.145949 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:11.156305 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a61398-550b-4440-b2c7-fb3305d8c2bc None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:11.192245 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760c87b1-80de-49fd-b6e9-d1155ed1edb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:11.192901 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c87b1-80de-49fd-b6e9-d1155ed1edb7 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:11.193118 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c87b1-80de-49fd-b6e9-d1155ed1edb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:11.193429 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c87b1-80de-49fd-b6e9-d1155ed1edb7 None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:11.194085 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760c87b1-80de-49fd-b6e9-d1155ed1edb7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:11.194395 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760c87b1-80de-49fd-b6e9-d1155ed1edb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:11.195788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760c87b1-80de-49fd-b6e9-d1155ed1edb7 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:11.208374 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 543/1063] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Oct 3 12:32:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:32:11.212394 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d020b-b085-4111-88fb-2dcd173e4c9b None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:11.232435 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:11.337680 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d020b-b085-4111-88fb-2dcd173e4c9b None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:11.376391 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d413126c-8dd4-43c4-93b8-4d92b645d32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:11.377167 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d413126c-8dd4-43c4-93b8-4d92b645d32b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:11.377351 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d413126c-8dd4-43c4-93b8-4d92b645d32b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:11.377795 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d413126c-8dd4-43c4-93b8-4d92b645d32b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:11.389659 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d413126c-8dd4-43c4-93b8-4d92b645d32b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:11.391231 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d413126c-8dd4-43c4-93b8-4d92b645d32b None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:11.406468 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-d413126c-8dd4-43c4-93b8-4d92b645d32b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:32:11.406468 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:32:11.406468 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:32:11.406468 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:32:11.406468 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:32:11.406468 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:32:11.406468 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:32:11.406468 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:32:11.406468 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:32:11.406468 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:32:11.406468 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:32:11.406468 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:32:11.406468 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:32:11.406468 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:32:11.406468 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:32:11.406468 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:32:11.412792 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:32:11.412792 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:32:11.412792 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:32:11.412792 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:32:11.412792 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:32:11.412792 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:32:11.412792 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:32:11.412792 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:32:11.412792 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:32:11.412792 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:32:11.412792 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:32:11.412792 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:32:11.412792 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:32:11.412792 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:32:11.412792 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:32:11.412792 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:32:11.412792 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:32:11.413694 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:32:11.413694 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:32:11.413694 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:32:11.413694 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 521/1064] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:32:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:32:11.415129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760c87b1-80de-49fd-b6e9-d1155ed1edb7 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:11.519918 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760c87b1-80de-49fd-b6e9-d1155ed1edb7 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:11.557037 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37bba08-3638-4db1-95fc-551ccdc78f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:11.557666 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37bba08-3638-4db1-95fc-551ccdc78f3e None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:11.557950 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37bba08-3638-4db1-95fc-551ccdc78f3e None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:11.558216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37bba08-3638-4db1-95fc-551ccdc78f3e None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:11.558872 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37bba08-3638-4db1-95fc-551ccdc78f3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:11.559187 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37bba08-3638-4db1-95fc-551ccdc78f3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:11.560610 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37bba08-3638-4db1-95fc-551ccdc78f3e None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:11.574532 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 544/1065] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:32:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:11.579934 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d413126c-8dd4-43c4-93b8-4d92b645d32b None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:11.648182 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:11.658979 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bdf98-42ba-4e32-857f-9f0de5ce09c5 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:11.659223 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bdf98-42ba-4e32-857f-9f0de5ce09c5 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:11.659490 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bdf98-42ba-4e32-857f-9f0de5ce09c5 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:11.660952 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 545/1066] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:32:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 12:32:11.665718 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10200a-f180-41a7-ac65-b53c5d243e61 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:11.666004 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10200a-f180-41a7-ac65-b53c5d243e61 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:11.666281 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10200a-f180-41a7-ac65-b53c5d243e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:11.683740 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d413126c-8dd4-43c4-93b8-4d92b645d32b None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:11.684523 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-ba10200a-f180-41a7-ac65-b53c5d243e61 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:32:11.685698 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-ba10200a-f180-41a7-ac65-b53c5d243e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:32:11.693254 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:11.718755 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41834d5a-8735-46e3-a60b-d78cfe0f32da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:11.719691 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41834d5a-8735-46e3-a60b-d78cfe0f32da None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:11.719953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41834d5a-8735-46e3-a60b-d78cfe0f32da None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:11.720248 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41834d5a-8735-46e3-a60b-d78cfe0f32da None admin] PATH_INFO: `/v3/users/nova` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:11.727852 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41834d5a-8735-46e3-a60b-d78cfe0f32da None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:11.729280 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41834d5a-8735-46e3-a60b-d78cfe0f32da None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:11.741095 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-41834d5a-8735-46e3-a60b-d78cfe0f32da None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 03 12:32:11.741095 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:32:11.741095 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:32:11.741095 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:32:11.741095 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:32:11.741095 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:32:11.741095 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:32:11.741095 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:32:11.741095 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:32:11.741095 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:32:11.741095 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:32:11.741095 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:32:11.741095 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 12:32:11.741095 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 12:32:11.741095 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 12:32:11.741095 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 12:32:11.742058 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:32:11.742058 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:32:11.742058 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 12:32:11.742058 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:32:11.742058 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 12:32:11.742058 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:32:11.742058 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:32:11.742058 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:32:11.742058 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:32:11.742058 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:32:11.742058 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:32:11.742058 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:32:11.742058 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:32:11.742058 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:32:11.742058 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:32:11.742058 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:32:11.742058 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:32:11.742058 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:32:11.743039 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:32:11.743039 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:32:11.743039 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 12:32:11.743039 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 12:32:11.743039 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 12:32:11.743039 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 12:32:11.743039 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 12:32:11.743039 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 12:32:11.743039 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 03 12:32:11.743039 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:32:11.747121 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 522/1067] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Oct 3 12:32:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:32:11.747899 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41834d5a-8735-46e3-a60b-d78cfe0f32da None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:11.759274 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 546/1068] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:32:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 12:32:11.847906 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41834d5a-8735-46e3-a60b-d78cfe0f32da None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:11.873971 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:11.882824 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e39334-6c92-4aa9-bcf3-a1f494f5e849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:11.887077 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e39334-6c92-4aa9-bcf3-a1f494f5e849 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:11.887077 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e39334-6c92-4aa9-bcf3-a1f494f5e849 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:11.887077 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e39334-6c92-4aa9-bcf3-a1f494f5e849 None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:11.887077 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e39334-6c92-4aa9-bcf3-a1f494f5e849 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:11.887077 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e39334-6c92-4aa9-bcf3-a1f494f5e849 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:11.887077 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e39334-6c92-4aa9-bcf3-a1f494f5e849 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:11.907743 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 523/1069] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Oct 3 12:32:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:32:11.908970 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba10200a-f180-41a7-ac65-b53c5d243e61 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:12.007049 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba10200a-f180-41a7-ac65-b53c5d243e61 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:12.027039 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e39334-6c92-4aa9-bcf3-a1f494f5e849 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:12.040888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d462df-38f4-4e68-8470-b51f82419c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:12.041524 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d462df-38f4-4e68-8470-b51f82419c8f None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:12.041804 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d462df-38f4-4e68-8470-b51f82419c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:12.042077 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d462df-38f4-4e68-8470-b51f82419c8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:12.051967 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d462df-38f4-4e68-8470-b51f82419c8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:12.054095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d462df-38f4-4e68-8470-b51f82419c8f None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:12.055847 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:12.070107 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-b4d462df-38f4-4e68-8470-b51f82419c8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:32:12.070107 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:32:12.070107 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:32:12.070107 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:32:12.070107 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:32:12.070107 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:32:12.070107 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:32:12.070107 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:32:12.070107 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:32:12.070107 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:32:12.070107 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:32:12.070107 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:32:12.070107 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:32:12.070107 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:32:12.070107 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:32:12.070107 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:32:12.071246 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:32:12.071246 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:32:12.071246 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:32:12.071246 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:32:12.071246 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:32:12.071246 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:32:12.071246 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:32:12.071246 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:32:12.071246 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:32:12.071246 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:32:12.071246 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:32:12.071246 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:32:12.071246 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:32:12.071246 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:32:12.071246 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:32:12.071246 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:32:12.071246 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:32:12.072378 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:32:12.072378 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:32:12.072378 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:32:12.074081 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 547/1070] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:32:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:32:12.079431 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d462df-38f4-4e68-8470-b51f82419c8f None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:12.139229 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e39334-6c92-4aa9-bcf3-a1f494f5e849 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:12.177020 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f36eaa9-4bbe-4cee-b663-28aa14fb6248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:12.177712 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36eaa9-4bbe-4cee-b663-28aa14fb6248 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:12.178270 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36eaa9-4bbe-4cee-b663-28aa14fb6248 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:12.178270 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36eaa9-4bbe-4cee-b663-28aa14fb6248 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:12.186810 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d462df-38f4-4e68-8470-b51f82419c8f None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:12.189763 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f36eaa9-4bbe-4cee-b663-28aa14fb6248 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:32:12.195515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-2f36eaa9-4bbe-4cee-b663-28aa14fb6248 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:32:12.195515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:32:12.195515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:32:12.195515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:32:12.195515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:32:12.195515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:32:12.195515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:32:12.195515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:32:12.195515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:32:12.195515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:32:12.195515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:32:12.195515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:32:12.195515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:32:12.195515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:32:12.195515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:32:12.195515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:32:12.196483 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:32:12.196483 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:32:12.196483 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:32:12.196483 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:32:12.196483 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:32:12.196483 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:32:12.196483 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:32:12.196483 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:32:12.196483 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:32:12.196483 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:32:12.196483 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:32:12.196483 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:32:12.196483 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:32:12.196483 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:32:12.196483 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:32:12.196483 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:32:12.197327 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:32:12.197327 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:32:12.197327 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:32:12.197327 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:32:12.197327 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:32:12.197327 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:32:12.197327 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:32:12.197327 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:32:12.197327 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:32:12.197327 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:32:12.197327 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:32:12.197327 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:32:12.197327 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 524/1071] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 12:32:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:32:12.202808 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f36eaa9-4bbe-4cee-b663-28aa14fb6248 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:12.226752 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e642cf28-84ba-4e88-b681-bed34c13ac3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:12.227467 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e642cf28-84ba-4e88-b681-bed34c13ac3b None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:12.227687 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e642cf28-84ba-4e88-b681-bed34c13ac3b None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:12.228087 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e642cf28-84ba-4e88-b681-bed34c13ac3b None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:12.228735 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e642cf28-84ba-4e88-b681-bed34c13ac3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:12.229045 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e642cf28-84ba-4e88-b681-bed34c13ac3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:12.230404 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e642cf28-84ba-4e88-b681-bed34c13ac3b None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:12.254601 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 548/1072] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:32:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:32:12.255537 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e642cf28-84ba-4e88-b681-bed34c13ac3b None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:12.255688 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:12.310392 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f36eaa9-4bbe-4cee-b663-28aa14fb6248 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:12.315675 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:12.355115 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06b4f9-1a7d-427b-b781-ae341dd4f260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:12.355821 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06b4f9-1a7d-427b-b781-ae341dd4f260 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:12.356079 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06b4f9-1a7d-427b-b781-ae341dd4f260 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:12.356304 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06b4f9-1a7d-427b-b781-ae341dd4f260 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:12.356963 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc06b4f9-1a7d-427b-b781-ae341dd4f260 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:12.357260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc06b4f9-1a7d-427b-b781-ae341dd4f260 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:12.358620 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc06b4f9-1a7d-427b-b781-ae341dd4f260 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:12.367914 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 525/1073] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:32:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:12.368058 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e642cf28-84ba-4e88-b681-bed34c13ac3b None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:12.373229 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06b4f9-1a7d-427b-b781-ae341dd4f260 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:12.405532 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49cc1ac-2c7b-4d0c-bf60-7cd2d1fd1b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:12.406303 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cc1ac-2c7b-4d0c-bf60-7cd2d1fd1b37 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:12.406588 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cc1ac-2c7b-4d0c-bf60-7cd2d1fd1b37 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:12.406857 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cc1ac-2c7b-4d0c-bf60-7cd2d1fd1b37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:12.414549 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49cc1ac-2c7b-4d0c-bf60-7cd2d1fd1b37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:12.416017 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49cc1ac-2c7b-4d0c-bf60-7cd2d1fd1b37 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:12.428287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-a49cc1ac-2c7b-4d0c-bf60-7cd2d1fd1b37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:32:12.428287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:32:12.428287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:32:12.428287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:32:12.428287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:32:12.428287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:32:12.428287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:32:12.428287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:32:12.428287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:32:12.428287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:32:12.428287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:32:12.428287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:32:12.428287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 12:32:12.428287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 12:32:12.428287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 12:32:12.429387 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 12:32:12.429387 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:32:12.429387 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:32:12.429387 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:32:12.429387 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:32:12.429387 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:32:12.429387 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:32:12.429387 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:32:12.429387 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:32:12.429387 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:32:12.429387 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:32:12.429387 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:32:12.429387 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:32:12.429387 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:32:12.429387 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:32:12.429387 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:32:12.429387 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:32:12.430442 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 12:32:12.430442 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 12:32:12.430442 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 12:32:12.430442 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 12:32:12.430442 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 12:32:12.430442 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 12:32:12.430442 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:32:12.430442 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:32:12.430442 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 549/1074] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:32:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:32:12.435107 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49cc1ac-2c7b-4d0c-bf60-7cd2d1fd1b37 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:12.500805 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06b4f9-1a7d-427b-b781-ae341dd4f260 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:12.510626 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:12.537631 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cde383-efcd-4783-ad6f-3285e2d723a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:12.538605 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cde383-efcd-4783-ad6f-3285e2d723a5 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:12.538882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cde383-efcd-4783-ad6f-3285e2d723a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:12.539460 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cde383-efcd-4783-ad6f-3285e2d723a5 None admin] PATH_INFO: `/v3/users/e6c7b23c70cc49a0a106e38e514ac54c` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:12.540767 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cde383-efcd-4783-ad6f-3285e2d723a5 None admin] RBAC: Authorizing `identity:get_user(user_id=e6c7b23c70cc49a0a106e38e514ac54c)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:12.542516 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cde383-efcd-4783-ad6f-3285e2d723a5 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:12.544138 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 526/1075] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:32:12 2022] GET /identity/v3/users/e6c7b23c70cc49a0a106e38e514ac54c => generated 285 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:32:12.546994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49cc1ac-2c7b-4d0c-bf60-7cd2d1fd1b37 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:12.550561 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cde383-efcd-4783-ad6f-3285e2d723a5 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:12.561691 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:12.581828 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a2ea45-e728-4e93-b81e-b53d6c7f33a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:12.582444 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a2ea45-e728-4e93-b81e-b53d6c7f33a0 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:12.582727 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a2ea45-e728-4e93-b81e-b53d6c7f33a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:12.583006 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a2ea45-e728-4e93-b81e-b53d6c7f33a0 None admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:12.583724 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a2ea45-e728-4e93-b81e-b53d6c7f33a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:12.584623 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a2ea45-e728-4e93-b81e-b53d6c7f33a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:12.586166 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a2ea45-e728-4e93-b81e-b53d6c7f33a0 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:12.599115 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 550/1076] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 12:32:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:12.604832 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a2ea45-e728-4e93-b81e-b53d6c7f33a0 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:12.661303 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cde383-efcd-4783-ad6f-3285e2d723a5 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:12.698690 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4427aa-02e9-46c9-82c6-c80f9bef0518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:12.699462 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4427aa-02e9-46c9-82c6-c80f9bef0518 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:12.699747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4427aa-02e9-46c9-82c6-c80f9bef0518 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:12.700025 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4427aa-02e9-46c9-82c6-c80f9bef0518 None admin] PATH_INFO: `/v3/projects/c73bdc87381d4b479b5c19adf5d3c06d` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:12.701138 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4427aa-02e9-46c9-82c6-c80f9bef0518 None admin] RBAC: Authorizing `identity:get_project(project_id=c73bdc87381d4b479b5c19adf5d3c06d)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:12.702585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4427aa-02e9-46c9-82c6-c80f9bef0518 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:12.704248 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 527/1077] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:32:12 2022] GET /identity/v3/projects/c73bdc87381d4b479b5c19adf5d3c06d => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:32:12.710557 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4427aa-02e9-46c9-82c6-c80f9bef0518 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:12.716250 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a2ea45-e728-4e93-b81e-b53d6c7f33a0 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:12.755370 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566fe8f-ab6a-4774-9c6c-a487b221e779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:12.756744 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566fe8f-ab6a-4774-9c6c-a487b221e779 None admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:12.757177 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566fe8f-ab6a-4774-9c6c-a487b221e779 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:12.757406 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566fe8f-ab6a-4774-9c6c-a487b221e779 None admin] PATH_INFO: `/v3/projects/4ea2fa8ad01240f7859182ebbe3051c7/users/990f5837994f4a088c656c9b33e9a0dc/roles/a7c026bed2224838b2a71641eb61b9d0` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:12.759628 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3566fe8f-ab6a-4774-9c6c-a487b221e779 None admin] RBAC: Authorizing `identity:create_grant(project_id=4ea2fa8ad01240f7859182ebbe3051c7, user_id=990f5837994f4a088c656c9b33e9a0dc, role_id=a7c026bed2224838b2a71641eb61b9d0)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:12.760242 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:12.761055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3566fe8f-ab6a-4774-9c6c-a487b221e779 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:12.814021 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4427aa-02e9-46c9-82c6-c80f9bef0518 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:12.846215 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061521c1-fbd5-4761-9228-5add878ab191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:12.846866 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061521c1-fbd5-4761-9228-5add878ab191 None admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:12.847157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061521c1-fbd5-4761-9228-5add878ab191 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:12.847456 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061521c1-fbd5-4761-9228-5add878ab191 None admin] PATH_INFO: `/v3/projects/c73bdc87381d4b479b5c19adf5d3c06d/users/e6c7b23c70cc49a0a106e38e514ac54c/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:12.849349 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061521c1-fbd5-4761-9228-5add878ab191 None admin] RBAC: Authorizing `identity:create_grant(project_id=c73bdc87381d4b479b5c19adf5d3c06d, user_id=e6c7b23c70cc49a0a106e38e514ac54c, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:12.850978 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 551/1078] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 12:32:12 2022] PUT /identity/v3/projects/4ea2fa8ad01240f7859182ebbe3051c7/users/990f5837994f4a088c656c9b33e9a0dc/roles/a7c026bed2224838b2a71641eb61b9d0 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:32:12.852265 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a49fd-6e7d-48c5-8c46-9db01d76a902 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:12.852553 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a49fd-6e7d-48c5-8c46-9db01d76a902 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:12.852827 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a49fd-6e7d-48c5-8c46-9db01d76a902 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:12.854068 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 552/1079] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:32:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:32:12.855847 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061521c1-fbd5-4761-9228-5add878ab191 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:12.864311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333afd3-37c9-4938-9e18-5e35b88192ce None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:12.864599 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333afd3-37c9-4938-9e18-5e35b88192ce None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:12.864876 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333afd3-37c9-4938-9e18-5e35b88192ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:12.886141 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-e333afd3-37c9-4938-9e18-5e35b88192ce None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:32:12.887401 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-e333afd3-37c9-4938-9e18-5e35b88192ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:32:12.918583 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:12.953363 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 528/1080] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 12:32:12 2022] PUT /identity/v3/projects/c73bdc87381d4b479b5c19adf5d3c06d/users/e6c7b23c70cc49a0a106e38e514ac54c/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:32:12.963180 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 553/1081] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:32:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 12:32:13.238166 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061521c1-fbd5-4761-9228-5add878ab191 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:13.343304 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061521c1-fbd5-4761-9228-5add878ab191 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:13.346852 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:13.377857 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81885208-55c0-4a03-8060-1831ca71e142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:13.380295 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81885208-55c0-4a03-8060-1831ca71e142 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:13.380295 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81885208-55c0-4a03-8060-1831ca71e142 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:13.380295 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81885208-55c0-4a03-8060-1831ca71e142 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:13.453470 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81885208-55c0-4a03-8060-1831ca71e142 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:13.454916 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81885208-55c0-4a03-8060-1831ca71e142 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:13.558262 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 529/1082] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:32:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 12:32:13.563764 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333afd3-37c9-4938-9e18-5e35b88192ce None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:13.663206 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333afd3-37c9-4938-9e18-5e35b88192ce None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:13.697509 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec363635-e106-429f-93f1-2eb829b73ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:13.698463 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec363635-e106-429f-93f1-2eb829b73ec4 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:13.698938 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec363635-e106-429f-93f1-2eb829b73ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:13.699412 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec363635-e106-429f-93f1-2eb829b73ec4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:13.709780 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec363635-e106-429f-93f1-2eb829b73ec4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:13.711461 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec363635-e106-429f-93f1-2eb829b73ec4 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:13.727798 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-ec363635-e106-429f-93f1-2eb829b73ec4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:32:13.727798 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:32:13.727798 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:32:13.727798 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:32:13.727798 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:32:13.727798 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:32:13.727798 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:32:13.727798 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:32:13.727798 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:32:13.727798 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:32:13.727798 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:32:13.727798 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:32:13.727798 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:32:13.727798 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:32:13.727798 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:32:13.727798 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:32:13.728825 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:32:13.728825 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:32:13.728825 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:32:13.728825 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:32:13.728825 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:32:13.728825 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:32:13.728825 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:32:13.728825 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:32:13.728825 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:32:13.728825 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:32:13.728825 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:32:13.728825 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:32:13.728825 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:32:13.728825 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:32:13.728825 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:32:13.728825 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:32:13.728825 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:32:13.729739 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:32:13.729739 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:32:13.729739 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:32:13.730168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 554/1083] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:32:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:32:13.736219 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81885208-55c0-4a03-8060-1831ca71e142 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:13.739837 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:13.847481 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81885208-55c0-4a03-8060-1831ca71e142 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:13.884164 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dda20a-3a68-497f-8408-adb75ae99ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:13.884770 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dda20a-3a68-497f-8408-adb75ae99ade None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:13.885043 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dda20a-3a68-497f-8408-adb75ae99ade None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:13.885309 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dda20a-3a68-497f-8408-adb75ae99ade None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:13.885938 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dda20a-3a68-497f-8408-adb75ae99ade None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:13.886233 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dda20a-3a68-497f-8408-adb75ae99ade None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:13.887786 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dda20a-3a68-497f-8408-adb75ae99ade None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:13.907354 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 530/1084] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:32:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:32:15.078465 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ccd23-8a0e-4c8c-832f-0091fbdfbb36 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:15.079065 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ccd23-8a0e-4c8c-832f-0091fbdfbb36 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:15.079065 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ccd23-8a0e-4c8c-832f-0091fbdfbb36 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:15.080415 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 555/1085] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:32:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:32:15.087007 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f26f2-a312-4204-8d21-465c034d95ed None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:15.087613 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f26f2-a312-4204-8d21-465c034d95ed None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:15.087613 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f26f2-a312-4204-8d21-465c034d95ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:15.106057 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-ff1f26f2-a312-4204-8d21-465c034d95ed None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:32:15.107230 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ff1f26f2-a312-4204-8d21-465c034d95ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:32:15.164740 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:15.172903 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 531/1086] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:32:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:32:15.188684 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818cd32-4b13-41b2-b1ff-c4f5e165e934 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:15.188961 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818cd32-4b13-41b2-b1ff-c4f5e165e934 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:15.189226 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818cd32-4b13-41b2-b1ff-c4f5e165e934 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:15.190534 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 556/1087] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:32:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:32:15.196340 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b264b-8d1b-4a4b-af30-9abfdcf9a04d None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:15.196631 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b264b-8d1b-4a4b-af30-9abfdcf9a04d None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:15.196903 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b264b-8d1b-4a4b-af30-9abfdcf9a04d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:15.215810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-e76b264b-8d1b-4a4b-af30-9abfdcf9a04d None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:32:15.218508 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e76b264b-8d1b-4a4b-af30-9abfdcf9a04d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:32:15.288053 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 532/1088] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:32:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:32:15.448022 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818cd32-4b13-41b2-b1ff-c4f5e165e934 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:15.466665 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:15.553032 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818cd32-4b13-41b2-b1ff-c4f5e165e934 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:15.561153 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b264b-8d1b-4a4b-af30-9abfdcf9a04d None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:15.586906 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f4597-1620-4fd1-a12d-ff04be955902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:15.587567 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f4597-1620-4fd1-a12d-ff04be955902 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:15.587853 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f4597-1620-4fd1-a12d-ff04be955902 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:15.588125 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f4597-1620-4fd1-a12d-ff04be955902 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:15.598264 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59f4597-1620-4fd1-a12d-ff04be955902 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:32:15.603853 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-c59f4597-1620-4fd1-a12d-ff04be955902 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 12:32:15.603853 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:32:15.603853 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:32:15.603853 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:32:15.603853 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:32:15.603853 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:32:15.603853 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:32:15.603853 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:32:15.603853 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:32:15.603853 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:32:15.603853 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:32:15.603853 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:32:15.603853 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:32:15.603853 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:32:15.603853 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:32:15.603853 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:32:15.604858 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:32:15.604858 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:32:15.604858 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:32:15.604858 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:32:15.604858 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:32:15.604858 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:32:15.604858 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:32:15.604858 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:32:15.604858 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:32:15.604858 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:32:15.604858 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:32:15.604858 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:32:15.604858 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:32:15.604858 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:32:15.604858 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:32:15.604858 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:32:15.605712 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:32:15.605712 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:32:15.605712 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:32:15.605712 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:32:15.605712 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:32:15.605712 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:32:15.605712 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:32:15.605712 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:32:15.605712 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:32:15.605712 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:32:15.605712 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 12:32:15.605712 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:32:15.605712 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 557/1089] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Oct 3 12:32:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:32:15.610841 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f4597-1620-4fd1-a12d-ff04be955902 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:15.664315 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b264b-8d1b-4a4b-af30-9abfdcf9a04d None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:15.685461 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:15.687469 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:15.698644 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22bc50a-523d-4501-9405-4e7be983de4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:15.699210 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22bc50a-523d-4501-9405-4e7be983de4c None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:15.699514 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22bc50a-523d-4501-9405-4e7be983de4c None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:15.699789 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22bc50a-523d-4501-9405-4e7be983de4c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:15.707714 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f4597-1620-4fd1-a12d-ff04be955902 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:15.710627 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22bc50a-523d-4501-9405-4e7be983de4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:32:15.715402 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-e22bc50a-523d-4501-9405-4e7be983de4c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:32:15.715402 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:32:15.715402 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:32:15.715402 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:32:15.715402 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:32:15.715402 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:32:15.715402 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:32:15.715402 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:32:15.715402 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:32:15.715402 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:32:15.715402 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:32:15.715402 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:32:15.715402 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:32:15.715402 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:32:15.715402 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:32:15.715402 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:32:15.716370 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:32:15.716370 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:32:15.716370 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:32:15.716370 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:32:15.716370 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:32:15.716370 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:32:15.716370 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:32:15.716370 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:32:15.716370 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:32:15.716370 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:32:15.716370 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:32:15.716370 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:32:15.716370 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:32:15.716370 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:32:15.716370 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:32:15.716370 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:32:15.717207 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:32:15.717207 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:32:15.717207 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:32:15.717207 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:32:15.717207 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:32:15.717207 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:32:15.717207 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:32:15.717207 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:32:15.717207 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:32:15.717207 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:32:15.717207 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:32:15.717207 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:32:15.717207 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 533/1090] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 12:32:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:32:15.723236 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22bc50a-523d-4501-9405-4e7be983de4c None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:15.741740 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed43daa-e554-40b2-a832-9de00a49aeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:15.742427 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed43daa-e554-40b2-a832-9de00a49aeed None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:15.742711 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed43daa-e554-40b2-a832-9de00a49aeed None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:15.742982 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed43daa-e554-40b2-a832-9de00a49aeed None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:15.743667 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed43daa-e554-40b2-a832-9de00a49aeed None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:15.743974 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed43daa-e554-40b2-a832-9de00a49aeed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:15.745314 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed43daa-e554-40b2-a832-9de00a49aeed None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:15.753888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 558/1091] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Oct 3 12:32:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:32:15.760232 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed43daa-e554-40b2-a832-9de00a49aeed None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:15.833471 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22bc50a-523d-4501-9405-4e7be983de4c None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:15.867891 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c4a7f2-372d-4fbf-86d1-9f5c36d7d502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:15.868492 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4a7f2-372d-4fbf-86d1-9f5c36d7d502 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:15.868763 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4a7f2-372d-4fbf-86d1-9f5c36d7d502 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:15.869036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4a7f2-372d-4fbf-86d1-9f5c36d7d502 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:15.869680 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c4a7f2-372d-4fbf-86d1-9f5c36d7d502 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:15.869976 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c4a7f2-372d-4fbf-86d1-9f5c36d7d502 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:15.870997 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed43daa-e554-40b2-a832-9de00a49aeed None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:15.871337 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c4a7f2-372d-4fbf-86d1-9f5c36d7d502 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:15.879059 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 534/1092] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:32:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:32:15.885611 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c4a7f2-372d-4fbf-86d1-9f5c36d7d502 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:15.910070 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03846120-7847-4dec-a618-48be82460a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:15.917351 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03846120-7847-4dec-a618-48be82460a8b None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:15.917351 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03846120-7847-4dec-a618-48be82460a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:15.917351 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03846120-7847-4dec-a618-48be82460a8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:15.919435 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:15.923242 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03846120-7847-4dec-a618-48be82460a8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:15.924721 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03846120-7847-4dec-a618-48be82460a8b None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:15.933264 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:15.939689 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-03846120-7847-4dec-a618-48be82460a8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:32:15.939689 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:32:15.939689 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:32:15.939689 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:32:15.939689 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:32:15.939689 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:32:15.939689 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:32:15.939689 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:32:15.939689 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:32:15.939689 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:32:15.939689 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:32:15.939689 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:32:15.939689 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:32:15.939689 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:32:15.939689 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:32:15.939689 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:32:15.940829 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:32:15.940829 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:32:15.940829 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:32:15.940829 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:32:15.940829 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:32:15.940829 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:32:15.940829 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:32:15.940829 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:32:15.940829 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:32:15.940829 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:32:15.940829 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:32:15.940829 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:32:15.940829 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:32:15.940829 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:32:15.940829 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:32:15.940829 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:32:15.940829 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:32:15.942218 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:32:15.942218 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:32:15.942218 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:32:15.942218 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 559/1093] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:32:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:32:15.946689 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03846120-7847-4dec-a618-48be82460a8b None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:15.986825 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c4a7f2-372d-4fbf-86d1-9f5c36d7d502 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:16.019364 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e6776-224f-4b65-bdc7-c1f56540c3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:16.019993 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e6776-224f-4b65-bdc7-c1f56540c3ad None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:16.023837 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e6776-224f-4b65-bdc7-c1f56540c3ad None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:16.023837 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e6776-224f-4b65-bdc7-c1f56540c3ad None admin] PATH_INFO: `/v3/users/e6c7b23c70cc49a0a106e38e514ac54c` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:16.023837 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e6776-224f-4b65-bdc7-c1f56540c3ad None admin] RBAC: Authorizing `identity:get_user(user_id=e6c7b23c70cc49a0a106e38e514ac54c)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:16.023837 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e6776-224f-4b65-bdc7-c1f56540c3ad None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:16.025227 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 535/1094] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:32:15 2022] GET /identity/v3/users/e6c7b23c70cc49a0a106e38e514ac54c => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:32:16.030996 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e6776-224f-4b65-bdc7-c1f56540c3ad None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:16.044913 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03846120-7847-4dec-a618-48be82460a8b None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:16.079463 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755de67b-db8b-403f-90f6-6bbd428ecdf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:16.080119 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755de67b-db8b-403f-90f6-6bbd428ecdf8 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:16.080338 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755de67b-db8b-403f-90f6-6bbd428ecdf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:16.080610 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755de67b-db8b-403f-90f6-6bbd428ecdf8 None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:16.081295 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755de67b-db8b-403f-90f6-6bbd428ecdf8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:16.081603 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755de67b-db8b-403f-90f6-6bbd428ecdf8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:16.083054 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755de67b-db8b-403f-90f6-6bbd428ecdf8 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:16.096851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 560/1095] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:32:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:32:16.104724 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755de67b-db8b-403f-90f6-6bbd428ecdf8 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:16.135770 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e6776-224f-4b65-bdc7-c1f56540c3ad None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:16.149377 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:16.161827 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:16.169051 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3ceef-177a-4c8f-9eda-f0b5ed775c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:16.169662 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3ceef-177a-4c8f-9eda-f0b5ed775c46 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:16.169943 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3ceef-177a-4c8f-9eda-f0b5ed775c46 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:16.170260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3ceef-177a-4c8f-9eda-f0b5ed775c46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:16.237239 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3ceef-177a-4c8f-9eda-f0b5ed775c46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:16.238595 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3ceef-177a-4c8f-9eda-f0b5ed775c46 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:16.331521 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755de67b-db8b-403f-90f6-6bbd428ecdf8 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:16.367198 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d90ee40-f77f-4c7c-b6a4-124b6182bad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:16.367851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90ee40-f77f-4c7c-b6a4-124b6182bad3 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:16.368126 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90ee40-f77f-4c7c-b6a4-124b6182bad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:16.368396 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90ee40-f77f-4c7c-b6a4-124b6182bad3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:16.375458 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d90ee40-f77f-4c7c-b6a4-124b6182bad3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:16.376890 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d90ee40-f77f-4c7c-b6a4-124b6182bad3 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:16.388566 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-0d90ee40-f77f-4c7c-b6a4-124b6182bad3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 03 12:32:16.388566 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:32:16.388566 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:32:16.388566 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:32:16.388566 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:32:16.388566 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:32:16.388566 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:32:16.388566 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:32:16.388566 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:32:16.388566 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:32:16.388566 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:32:16.388566 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:32:16.388566 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 12:32:16.388566 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 12:32:16.388566 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 12:32:16.388566 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 12:32:16.389639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:32:16.389639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:32:16.389639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 12:32:16.389639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:32:16.389639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 12:32:16.389639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:32:16.389639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:32:16.389639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:32:16.389639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:32:16.389639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:32:16.389639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:32:16.389639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:32:16.389639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:32:16.389639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:32:16.389639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:32:16.389639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:32:16.389639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:32:16.389639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:32:16.390605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:32:16.390605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:32:16.390605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 12:32:16.390605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 12:32:16.390605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 12:32:16.390605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 12:32:16.390605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 12:32:16.390605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 12:32:16.390605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 03 12:32:16.390605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:32:16.390605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 561/1096] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Oct 3 12:32:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:32:16.395577 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d90ee40-f77f-4c7c-b6a4-124b6182bad3 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:16.403061 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 536/1097] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:32:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 12:32:16.406436 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3ceef-177a-4c8f-9eda-f0b5ed775c46 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:16.488367 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:16.573122 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d90ee40-f77f-4c7c-b6a4-124b6182bad3 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:16.576255 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3ceef-177a-4c8f-9eda-f0b5ed775c46 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:16.583408 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:16.608038 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875f3d9-2fcd-493c-a267-8458703c61b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:16.608645 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875f3d9-2fcd-493c-a267-8458703c61b0 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:16.608924 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875f3d9-2fcd-493c-a267-8458703c61b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:16.609196 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875f3d9-2fcd-493c-a267-8458703c61b0 None admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:16.609915 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875f3d9-2fcd-493c-a267-8458703c61b0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:16.610221 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875f3d9-2fcd-493c-a267-8458703c61b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:16.611661 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875f3d9-2fcd-493c-a267-8458703c61b0 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:16.611794 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea8f2ba-9db1-4408-8a83-adbb293910eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:16.612416 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8f2ba-9db1-4408-8a83-adbb293910eb None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:16.612690 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8f2ba-9db1-4408-8a83-adbb293910eb None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:16.612958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8f2ba-9db1-4408-8a83-adbb293910eb None admin] PATH_INFO: `/v3/projects/c73bdc87381d4b479b5c19adf5d3c06d` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:16.614042 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea8f2ba-9db1-4408-8a83-adbb293910eb None admin] RBAC: Authorizing `identity:get_project(project_id=c73bdc87381d4b479b5c19adf5d3c06d)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:16.615520 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea8f2ba-9db1-4408-8a83-adbb293910eb None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:16.617206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 537/1098] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:32:16 2022] GET /identity/v3/projects/c73bdc87381d4b479b5c19adf5d3c06d => generated 311 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:32:16.623240 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea8f2ba-9db1-4408-8a83-adbb293910eb None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:16.635061 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 562/1099] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Oct 3 12:32:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:32:16.636446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875f3d9-2fcd-493c-a267-8458703c61b0 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:16.708651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:16.728693 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea8f2ba-9db1-4408-8a83-adbb293910eb None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:16.735711 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875f3d9-2fcd-493c-a267-8458703c61b0 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:16.764931 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef503c8-5581-4b39-b9dd-847b7a516100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:16.765541 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef503c8-5581-4b39-b9dd-847b7a516100 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:16.765875 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef503c8-5581-4b39-b9dd-847b7a516100 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:16.766265 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef503c8-5581-4b39-b9dd-847b7a516100 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:16.766987 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef503c8-5581-4b39-b9dd-847b7a516100 None admin] RBAC: Adding query filter params (role.id=7933c45dcdd54ef0a98018ba8a8bddaa, scope.project.id=c73bdc87381d4b479b5c19adf5d3c06d, user.id=e6c7b23c70cc49a0a106e38e514ac54c) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:16.767306 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef503c8-5581-4b39-b9dd-847b7a516100 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:16.768741 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef503c8-5581-4b39-b9dd-847b7a516100 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:16.769747 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b184a958-0c1a-4c2d-affb-35bfc472ab86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:16.770359 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184a958-0c1a-4c2d-affb-35bfc472ab86 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:16.770630 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184a958-0c1a-4c2d-affb-35bfc472ab86 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:16.770898 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184a958-0c1a-4c2d-affb-35bfc472ab86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:16.787667 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 538/1100] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:32:16 2022] GET /identity/v3/role_assignments?user.id=e6c7b23c70cc49a0a106e38e514ac54c&scope.project.id=c73bdc87381d4b479b5c19adf5d3c06d&role.id=7933c45dcdd54ef0a98018ba8a8bddaa => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:32:16.808393 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:16.845528 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b184a958-0c1a-4c2d-affb-35bfc472ab86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:16.846996 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b184a958-0c1a-4c2d-affb-35bfc472ab86 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:16.951188 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 563/1101] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:32:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 12:32:16.957140 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef503c8-5581-4b39-b9dd-847b7a516100 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:17.060237 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef503c8-5581-4b39-b9dd-847b7a516100 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:17.092971 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75cfe7-15a5-4a0e-bb6c-8fa953591adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:17.093598 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75cfe7-15a5-4a0e-bb6c-8fa953591adf None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:17.093872 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75cfe7-15a5-4a0e-bb6c-8fa953591adf None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:17.094138 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75cfe7-15a5-4a0e-bb6c-8fa953591adf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:17.104767 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a75cfe7-15a5-4a0e-bb6c-8fa953591adf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:17.106194 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a75cfe7-15a5-4a0e-bb6c-8fa953591adf None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:17.112294 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:17.120642 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-3a75cfe7-15a5-4a0e-bb6c-8fa953591adf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:32:17.120642 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:32:17.120642 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:32:17.120642 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:32:17.120642 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:32:17.120642 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:32:17.120642 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:32:17.120642 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:32:17.120642 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:32:17.120642 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:32:17.120642 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:32:17.120642 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:32:17.120642 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:32:17.120642 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:32:17.120642 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:32:17.120642 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:32:17.127100 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:32:17.127100 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:32:17.127100 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:32:17.127100 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:32:17.127100 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:32:17.127100 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:32:17.127100 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:32:17.127100 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:32:17.127100 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:32:17.127100 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:32:17.127100 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:32:17.127100 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:32:17.127100 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:32:17.127100 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:32:17.127100 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:32:17.127100 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:32:17.127100 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:32:17.128040 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:32:17.128040 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:32:17.128040 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:32:17.128040 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 539/1102] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:32:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:32:17.128262 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b184a958-0c1a-4c2d-affb-35bfc472ab86 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:17.193185 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:17.225968 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b184a958-0c1a-4c2d-affb-35bfc472ab86 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:17.262307 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131dc25d-81fa-4b1d-af39-fb19734a7400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:17.262912 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131dc25d-81fa-4b1d-af39-fb19734a7400 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:17.263187 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131dc25d-81fa-4b1d-af39-fb19734a7400 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:17.263488 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131dc25d-81fa-4b1d-af39-fb19734a7400 None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:17.264118 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131dc25d-81fa-4b1d-af39-fb19734a7400 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:17.264415 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131dc25d-81fa-4b1d-af39-fb19734a7400 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:17.265738 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131dc25d-81fa-4b1d-af39-fb19734a7400 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:17.277117 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 564/1103] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:32:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:32:17.282751 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75cfe7-15a5-4a0e-bb6c-8fa953591adf None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:17.382237 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75cfe7-15a5-4a0e-bb6c-8fa953591adf None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:17.416474 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd07ff68-2df1-427d-8520-bf2ff0646697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:17.417109 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07ff68-2df1-427d-8520-bf2ff0646697 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:17.417385 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07ff68-2df1-427d-8520-bf2ff0646697 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:17.417654 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07ff68-2df1-427d-8520-bf2ff0646697 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:17.424430 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd07ff68-2df1-427d-8520-bf2ff0646697 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:17.425844 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd07ff68-2df1-427d-8520-bf2ff0646697 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:17.437175 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-dd07ff68-2df1-427d-8520-bf2ff0646697 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:32:17.437175 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:32:17.437175 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:32:17.437175 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:32:17.437175 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:32:17.437175 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:32:17.437175 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:32:17.437175 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:32:17.437175 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:32:17.437175 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:32:17.437175 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:32:17.437175 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:32:17.437175 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 12:32:17.437175 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 12:32:17.437175 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 12:32:17.438206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 12:32:17.438206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:32:17.438206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:32:17.438206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:32:17.438206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:32:17.438206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:32:17.438206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:32:17.438206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:32:17.438206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:32:17.438206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:32:17.438206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:32:17.438206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:32:17.438206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:32:17.438206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:32:17.438206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:32:17.438206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:32:17.438206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:32:17.439284 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 12:32:17.439284 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 12:32:17.439284 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 12:32:17.439284 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 12:32:17.439284 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 12:32:17.439284 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 12:32:17.439284 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:32:17.439284 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:32:17.439284 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 540/1104] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:32:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:32:17.444711 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131dc25d-81fa-4b1d-af39-fb19734a7400 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:17.540686 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131dc25d-81fa-4b1d-af39-fb19734a7400 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:17.566918 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:17.574961 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e4b9a-3fc7-4308-858f-697597be0ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:17.575607 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e4b9a-3fc7-4308-858f-697597be0ab2 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:17.575887 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e4b9a-3fc7-4308-858f-697597be0ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:17.576156 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e4b9a-3fc7-4308-858f-697597be0ab2 None admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:17.576844 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33e4b9a-3fc7-4308-858f-697597be0ab2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:17.577146 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33e4b9a-3fc7-4308-858f-697597be0ab2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:17.578551 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33e4b9a-3fc7-4308-858f-697597be0ab2 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:17.590234 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 565/1105] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 12:32:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:17.596504 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd07ff68-2df1-427d-8520-bf2ff0646697 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:17.654147 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:17.695966 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd07ff68-2df1-427d-8520-bf2ff0646697 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:17.730920 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89b6654-7592-429d-9577-a3fb69112d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:17.731570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b6654-7592-429d-9577-a3fb69112d80 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:17.731849 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b6654-7592-429d-9577-a3fb69112d80 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:17.732188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b6654-7592-429d-9577-a3fb69112d80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:17.732951 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89b6654-7592-429d-9577-a3fb69112d80 None admin] RBAC: Adding query filter params (role.id=a7c026bed2224838b2a71641eb61b9d0, scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7, user.id=990f5837994f4a088c656c9b33e9a0dc) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:17.733255 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89b6654-7592-429d-9577-a3fb69112d80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:17.734714 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89b6654-7592-429d-9577-a3fb69112d80 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:17.741525 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 541/1106] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:32:17 2022] GET /identity/v3/role_assignments?user.id=990f5837994f4a088c656c9b33e9a0dc&scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7&role.id=a7c026bed2224838b2a71641eb61b9d0 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:19.074650 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706dc19-cb5a-44e9-9d39-842de4984db6 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:19.075206 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706dc19-cb5a-44e9-9d39-842de4984db6 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:19.075206 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706dc19-cb5a-44e9-9d39-842de4984db6 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:19.076595 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 566/1107] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:32:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:32:19.081739 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d550f1-c69e-4a34-b3da-d0c4d8a60149 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:19.082183 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d550f1-c69e-4a34-b3da-d0c4d8a60149 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:19.082283 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d550f1-c69e-4a34-b3da-d0c4d8a60149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:19.101084 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-15d550f1-c69e-4a34-b3da-d0c4d8a60149 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:32:19.102258 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-15d550f1-c69e-4a34-b3da-d0c4d8a60149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:32:19.166788 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 542/1108] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:32:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:32:19.483566 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d706dc19-cb5a-44e9-9d39-842de4984db6 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:19.602280 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d706dc19-cb5a-44e9-9d39-842de4984db6 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:19.643426 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4714f2-6f47-4cf4-8028-50e11da73f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:19.644068 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4714f2-6f47-4cf4-8028-50e11da73f53 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:19.644348 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4714f2-6f47-4cf4-8028-50e11da73f53 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:19.644621 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4714f2-6f47-4cf4-8028-50e11da73f53 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:19.645733 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4714f2-6f47-4cf4-8028-50e11da73f53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:19.647831 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4714f2-6f47-4cf4-8028-50e11da73f53 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:19.649421 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 567/1109] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:32:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:19.655298 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d550f1-c69e-4a34-b3da-d0c4d8a60149 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:19.715970 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:19.764433 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d550f1-c69e-4a34-b3da-d0c4d8a60149 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:19.810872 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764d8bf1-9b04-49c3-84a5-91542cafb140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:19.811513 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d8bf1-9b04-49c3-84a5-91542cafb140 None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:19.811788 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d8bf1-9b04-49c3-84a5-91542cafb140 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:19.812053 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d8bf1-9b04-49c3-84a5-91542cafb140 None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:19.812705 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764d8bf1-9b04-49c3-84a5-91542cafb140 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:19.814119 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764d8bf1-9b04-49c3-84a5-91542cafb140 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:19.925410 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 543/1110] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Oct 3 12:32:19 2022] POST /identity/v3/users => generated 286 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:32:22.135097 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d461a49-1920-48b3-b982-552d7c0b3e04 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:22.135675 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d461a49-1920-48b3-b982-552d7c0b3e04 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:22.135675 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d461a49-1920-48b3-b982-552d7c0b3e04 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:22.137006 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 568/1111] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:32:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:32:22.142254 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35efb8f-7847-4cae-b24d-a9dc99fe997d None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:22.142686 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35efb8f-7847-4cae-b24d-a9dc99fe997d None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:22.142804 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35efb8f-7847-4cae-b24d-a9dc99fe997d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:22.161309 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-c35efb8f-7847-4cae-b24d-a9dc99fe997d None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:32:22.163339 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-c35efb8f-7847-4cae-b24d-a9dc99fe997d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:32:22.230777 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 544/1112] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:32:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:32:22.496927 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d461a49-1920-48b3-b982-552d7c0b3e04 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:22.551941 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:22.595153 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d461a49-1920-48b3-b982-552d7c0b3e04 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:22.627545 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5cefd-1ef9-4892-b3a6-095371f390c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:22.628164 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5cefd-1ef9-4892-b3a6-095371f390c0 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:22.628442 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5cefd-1ef9-4892-b3a6-095371f390c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:22.628711 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5cefd-1ef9-4892-b3a6-095371f390c0 None admin] PATH_INFO: `/v3/roles/73b935a1ebc448899bbfc2381168d1cd` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:22.630069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf5cefd-1ef9-4892-b3a6-095371f390c0 None admin] RBAC: Authorizing `identity:get_role(role_id=73b935a1ebc448899bbfc2381168d1cd)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:22.631611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf5cefd-1ef9-4892-b3a6-095371f390c0 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:22.632907 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 569/1113] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:32:22 2022] GET /identity/v3/roles/73b935a1ebc448899bbfc2381168d1cd => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:22.638678 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35efb8f-7847-4cae-b24d-a9dc99fe997d None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:22.677269 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:22.735001 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35efb8f-7847-4cae-b24d-a9dc99fe997d None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:22.767926 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d455b2-41fc-41be-bf27-f6bddf071743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:22.768554 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d455b2-41fc-41be-bf27-f6bddf071743 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:22.768835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d455b2-41fc-41be-bf27-f6bddf071743 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:22.769111 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d455b2-41fc-41be-bf27-f6bddf071743 None admin] PATH_INFO: `/v3/users/73f62b5bb8d54d6589f70555ade6ffea` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:22.782304 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d455b2-41fc-41be-bf27-f6bddf071743 None admin] RBAC: Authorizing `identity:get_user(user_id=73f62b5bb8d54d6589f70555ade6ffea)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:22.783814 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d455b2-41fc-41be-bf27-f6bddf071743 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:22.785399 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 545/1114] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:32:22 2022] GET /identity/v3/users/73f62b5bb8d54d6589f70555ade6ffea => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:22.791251 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5cefd-1ef9-4892-b3a6-095371f390c0 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:22.885618 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5cefd-1ef9-4892-b3a6-095371f390c0 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:22.907570 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:22.918147 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00400e9-f518-4018-84df-32e0144521b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:22.918736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00400e9-f518-4018-84df-32e0144521b2 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:22.919012 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00400e9-f518-4018-84df-32e0144521b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:22.919281 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00400e9-f518-4018-84df-32e0144521b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:22.984656 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00400e9-f518-4018-84df-32e0144521b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:22.986084 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00400e9-f518-4018-84df-32e0144521b2 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:23.079257 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 570/1115] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:32:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 12:32:23.085154 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d455b2-41fc-41be-bf27-f6bddf071743 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:23.175971 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:23.182091 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d455b2-41fc-41be-bf27-f6bddf071743 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:23.214653 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2724d3-d5ec-4498-8c62-efbdf8ab256c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:23.215272 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2724d3-d5ec-4498-8c62-efbdf8ab256c None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:23.215577 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2724d3-d5ec-4498-8c62-efbdf8ab256c None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:23.215850 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2724d3-d5ec-4498-8c62-efbdf8ab256c None admin] PATH_INFO: `/v3/projects/c73bdc87381d4b479b5c19adf5d3c06d` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:23.216938 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2724d3-d5ec-4498-8c62-efbdf8ab256c None admin] RBAC: Authorizing `identity:get_project(project_id=c73bdc87381d4b479b5c19adf5d3c06d)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:23.218353 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2724d3-d5ec-4498-8c62-efbdf8ab256c None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:23.220494 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 546/1116] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:32:23 2022] GET /identity/v3/projects/c73bdc87381d4b479b5c19adf5d3c06d => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:32:23.227346 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00400e9-f518-4018-84df-32e0144521b2 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:23.252231 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:23.326144 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00400e9-f518-4018-84df-32e0144521b2 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:23.358686 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb9fa0-da87-4760-a2d3-b1920ad0ab6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:23.359224 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb9fa0-da87-4760-a2d3-b1920ad0ab6f None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:23.359530 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb9fa0-da87-4760-a2d3-b1920ad0ab6f None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:23.359801 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb9fa0-da87-4760-a2d3-b1920ad0ab6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:23.360509 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb9fa0-da87-4760-a2d3-b1920ad0ab6f None admin] RBAC: Adding query filter params (role.id=73b935a1ebc448899bbfc2381168d1cd, scope.project.id=c73bdc87381d4b479b5c19adf5d3c06d, user.id=73f62b5bb8d54d6589f70555ade6ffea) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:23.360816 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb9fa0-da87-4760-a2d3-b1920ad0ab6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:23.362212 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb9fa0-da87-4760-a2d3-b1920ad0ab6f None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:23.368009 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 571/1117] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:32:23 2022] GET /identity/v3/role_assignments?user.id=73f62b5bb8d54d6589f70555ade6ffea&scope.project.id=c73bdc87381d4b479b5c19adf5d3c06d&role.id=73b935a1ebc448899bbfc2381168d1cd => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:32:25.577763 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db00961-2113-437c-8348-2804af4c7b48 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:25.578279 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db00961-2113-437c-8348-2804af4c7b48 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:25.578279 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db00961-2113-437c-8348-2804af4c7b48 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:25.579592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 547/1118] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:32:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:32:25.585493 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858d577-3fef-48c6-9fe8-3ae94b5919ef None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:25.585881 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858d577-3fef-48c6-9fe8-3ae94b5919ef None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:25.586031 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858d577-3fef-48c6-9fe8-3ae94b5919ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:25.603893 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-d858d577-3fef-48c6-9fe8-3ae94b5919ef None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:32:25.605067 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-d858d577-3fef-48c6-9fe8-3ae94b5919ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:32:25.669213 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 572/1119] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:32:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:32:25.940867 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db00961-2113-437c-8348-2804af4c7b48 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:26.039118 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db00961-2113-437c-8348-2804af4c7b48 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:26.071962 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29a2e8a-2cb5-42d4-a17c-18e2628ecfc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:26.072575 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a2e8a-2cb5-42d4-a17c-18e2628ecfc5 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:26.072852 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a2e8a-2cb5-42d4-a17c-18e2628ecfc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:26.073121 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a2e8a-2cb5-42d4-a17c-18e2628ecfc5 None admin] PATH_INFO: `/v3/roles/73b935a1ebc448899bbfc2381168d1cd` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:26.074481 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29a2e8a-2cb5-42d4-a17c-18e2628ecfc5 None admin] RBAC: Authorizing `identity:get_role(role_id=73b935a1ebc448899bbfc2381168d1cd)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:26.075872 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29a2e8a-2cb5-42d4-a17c-18e2628ecfc5 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:26.077146 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 548/1120] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:32:25 2022] GET /identity/v3/roles/73b935a1ebc448899bbfc2381168d1cd => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:26.082828 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d858d577-3fef-48c6-9fe8-3ae94b5919ef None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:26.101832 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:26.177566 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d858d577-3fef-48c6-9fe8-3ae94b5919ef None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:26.210709 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b452d-d1ab-4eab-853e-d4573924024f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:26.211272 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b452d-d1ab-4eab-853e-d4573924024f None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:26.211584 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b452d-d1ab-4eab-853e-d4573924024f None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:26.211857 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b452d-d1ab-4eab-853e-d4573924024f None admin] PATH_INFO: `/v3/users/73f62b5bb8d54d6589f70555ade6ffea` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:26.212959 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b452d-d1ab-4eab-853e-d4573924024f None admin] RBAC: Authorizing `identity:get_user(user_id=73f62b5bb8d54d6589f70555ade6ffea)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:26.214366 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b452d-d1ab-4eab-853e-d4573924024f None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:26.215954 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 573/1121] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:32:26 2022] GET /identity/v3/users/73f62b5bb8d54d6589f70555ade6ffea => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:26.221921 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29a2e8a-2cb5-42d4-a17c-18e2628ecfc5 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:26.253425 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:26.322906 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29a2e8a-2cb5-42d4-a17c-18e2628ecfc5 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:26.356194 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e4f1b-d338-44d6-bd31-71815bb5386c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:26.356810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e4f1b-d338-44d6-bd31-71815bb5386c None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:26.357084 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e4f1b-d338-44d6-bd31-71815bb5386c None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:26.357351 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e4f1b-d338-44d6-bd31-71815bb5386c None admin] PATH_INFO: `/v3/projects/c73bdc87381d4b479b5c19adf5d3c06d` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:26.358427 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e4f1b-d338-44d6-bd31-71815bb5386c None admin] RBAC: Authorizing `identity:get_project(project_id=c73bdc87381d4b479b5c19adf5d3c06d)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:26.359868 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e4f1b-d338-44d6-bd31-71815bb5386c None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:26.361466 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 549/1122] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:32:26 2022] GET /identity/v3/projects/c73bdc87381d4b479b5c19adf5d3c06d => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:26.367911 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b452d-d1ab-4eab-853e-d4573924024f None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:26.450992 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:26.465106 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b452d-d1ab-4eab-853e-d4573924024f None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:26.498652 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263689f2-d1f6-497e-a7c9-9be3d8e19b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:26.499299 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263689f2-d1f6-497e-a7c9-9be3d8e19b0e None admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:26.499608 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263689f2-d1f6-497e-a7c9-9be3d8e19b0e None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:26.499879 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263689f2-d1f6-497e-a7c9-9be3d8e19b0e None admin] PATH_INFO: `/v3/projects/c73bdc87381d4b479b5c19adf5d3c06d/users/73f62b5bb8d54d6589f70555ade6ffea/roles/73b935a1ebc448899bbfc2381168d1cd` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:26.501952 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263689f2-d1f6-497e-a7c9-9be3d8e19b0e None admin] RBAC: Authorizing `identity:create_grant(project_id=c73bdc87381d4b479b5c19adf5d3c06d, user_id=73f62b5bb8d54d6589f70555ade6ffea, role_id=73b935a1ebc448899bbfc2381168d1cd)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:26.503398 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263689f2-d1f6-497e-a7c9-9be3d8e19b0e None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:26.575023 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 574/1123] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 12:32:26 2022] PUT /identity/v3/projects/c73bdc87381d4b479b5c19adf5d3c06d/users/73f62b5bb8d54d6589f70555ade6ffea/roles/73b935a1ebc448899bbfc2381168d1cd => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:32:28.702881 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f97a5a-83b3-4a11-bf5c-0864785fdc1e None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:28.703407 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f97a5a-83b3-4a11-bf5c-0864785fdc1e None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:28.703407 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f97a5a-83b3-4a11-bf5c-0864785fdc1e None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:28.704707 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 550/1124] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:32:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:32:28.709934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff144fe-7d88-45b9-ba3b-cf52a850bc42 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:28.710364 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff144fe-7d88-45b9-ba3b-cf52a850bc42 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:28.710465 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff144fe-7d88-45b9-ba3b-cf52a850bc42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:28.729702 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-bff144fe-7d88-45b9-ba3b-cf52a850bc42 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:32:28.730864 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-bff144fe-7d88-45b9-ba3b-cf52a850bc42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:32:28.794915 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 575/1125] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:32:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:32:29.069132 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f97a5a-83b3-4a11-bf5c-0864785fdc1e None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:29.164568 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:29.168288 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f97a5a-83b3-4a11-bf5c-0864785fdc1e None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:29.201589 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76597bf5-5ef2-448d-8273-1b9d77ff5880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:29.202206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76597bf5-5ef2-448d-8273-1b9d77ff5880 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:29.202483 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76597bf5-5ef2-448d-8273-1b9d77ff5880 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:29.202760 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76597bf5-5ef2-448d-8273-1b9d77ff5880 None admin] PATH_INFO: `/v3/roles/73b935a1ebc448899bbfc2381168d1cd` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:29.204710 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76597bf5-5ef2-448d-8273-1b9d77ff5880 None admin] RBAC: Authorizing `identity:get_role(role_id=73b935a1ebc448899bbfc2381168d1cd)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:29.206055 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76597bf5-5ef2-448d-8273-1b9d77ff5880 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:29.207372 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 551/1126] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:32:29 2022] GET /identity/v3/roles/73b935a1ebc448899bbfc2381168d1cd => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:29.213866 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff144fe-7d88-45b9-ba3b-cf52a850bc42 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:29.293906 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:29.316413 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff144fe-7d88-45b9-ba3b-cf52a850bc42 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:29.353561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd79558-c520-4ab9-b406-6ad66d353db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:29.354128 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd79558-c520-4ab9-b406-6ad66d353db1 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:29.354411 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd79558-c520-4ab9-b406-6ad66d353db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:29.354684 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd79558-c520-4ab9-b406-6ad66d353db1 None admin] PATH_INFO: `/v3/users/73f62b5bb8d54d6589f70555ade6ffea` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:29.355825 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd79558-c520-4ab9-b406-6ad66d353db1 None admin] RBAC: Authorizing `identity:get_user(user_id=73f62b5bb8d54d6589f70555ade6ffea)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:29.357259 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd79558-c520-4ab9-b406-6ad66d353db1 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:29.358840 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 576/1127] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:32:29 2022] GET /identity/v3/users/73f62b5bb8d54d6589f70555ade6ffea => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:29.364718 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76597bf5-5ef2-448d-8273-1b9d77ff5880 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:29.462911 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76597bf5-5ef2-448d-8273-1b9d77ff5880 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:29.496380 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6470faa-5102-4f15-b644-f34cad967e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:29.496984 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6470faa-5102-4f15-b644-f34cad967e32 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:29.497261 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6470faa-5102-4f15-b644-f34cad967e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:29.497531 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6470faa-5102-4f15-b644-f34cad967e32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:29.531185 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:29.563223 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6470faa-5102-4f15-b644-f34cad967e32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:29.564679 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6470faa-5102-4f15-b644-f34cad967e32 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:29.662482 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 552/1128] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:32:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 12:32:29.668427 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd79558-c520-4ab9-b406-6ad66d353db1 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:29.764039 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd79558-c520-4ab9-b406-6ad66d353db1 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:29.796036 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e05f1-36f6-4512-b564-21e03ee2c946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:29.796655 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e05f1-36f6-4512-b564-21e03ee2c946 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:29.796933 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e05f1-36f6-4512-b564-21e03ee2c946 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:29.797203 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e05f1-36f6-4512-b564-21e03ee2c946 None admin] PATH_INFO: `/v3/projects/c73bdc87381d4b479b5c19adf5d3c06d` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:29.798287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084e05f1-36f6-4512-b564-21e03ee2c946 None admin] RBAC: Authorizing `identity:get_project(project_id=c73bdc87381d4b479b5c19adf5d3c06d)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:29.799789 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084e05f1-36f6-4512-b564-21e03ee2c946 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:29.801399 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 577/1129] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:32:29 2022] GET /identity/v3/projects/c73bdc87381d4b479b5c19adf5d3c06d => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:29.808042 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6470faa-5102-4f15-b644-f34cad967e32 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:29.872735 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:29.906989 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6470faa-5102-4f15-b644-f34cad967e32 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:29.951991 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b414221-b079-471b-9e4e-b59f7b797b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:29.952524 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b414221-b079-471b-9e4e-b59f7b797b30 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:29.952803 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b414221-b079-471b-9e4e-b59f7b797b30 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:29.953080 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b414221-b079-471b-9e4e-b59f7b797b30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:29.953804 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b414221-b079-471b-9e4e-b59f7b797b30 None admin] RBAC: Adding query filter params (role.id=73b935a1ebc448899bbfc2381168d1cd, scope.project.id=c73bdc87381d4b479b5c19adf5d3c06d, user.id=73f62b5bb8d54d6589f70555ade6ffea) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:29.954110 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b414221-b079-471b-9e4e-b59f7b797b30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:29.955552 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b414221-b079-471b-9e4e-b59f7b797b30 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:29.961420 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 553/1130] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:32:29 2022] GET /identity/v3/role_assignments?user.id=73f62b5bb8d54d6589f70555ade6ffea&scope.project.id=c73bdc87381d4b479b5c19adf5d3c06d&role.id=73b935a1ebc448899bbfc2381168d1cd => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:32.116320 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d39bc5-4948-4e3b-80e3-ad328e785101 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:32.116895 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d39bc5-4948-4e3b-80e3-ad328e785101 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:32.116895 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d39bc5-4948-4e3b-80e3-ad328e785101 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:32.118202 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 578/1131] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:32:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:32:32.123521 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e1e69-d351-4df2-9247-72a2faa1a6d2 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:32.123903 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e1e69-d351-4df2-9247-72a2faa1a6d2 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:32.124110 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e1e69-d351-4df2-9247-72a2faa1a6d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:32.143399 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-743e1e69-d351-4df2-9247-72a2faa1a6d2 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:32:32.144572 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-743e1e69-d351-4df2-9247-72a2faa1a6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:32:32.208350 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 554/1132] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:32:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:32:32.476690 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d39bc5-4948-4e3b-80e3-ad328e785101 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:32.491849 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:32.581364 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d39bc5-4948-4e3b-80e3-ad328e785101 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:32.614960 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e3cbfc-d823-400e-8e84-098a23420c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:32.615632 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e3cbfc-d823-400e-8e84-098a23420c00 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:32.615910 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e3cbfc-d823-400e-8e84-098a23420c00 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:32.616179 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e3cbfc-d823-400e-8e84-098a23420c00 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:32.617258 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e3cbfc-d823-400e-8e84-098a23420c00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:32.618660 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e3cbfc-d823-400e-8e84-098a23420c00 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:32.620231 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 579/1133] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:32:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:32.626019 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e1e69-d351-4df2-9247-72a2faa1a6d2 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:32.685173 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:32.723445 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e1e69-d351-4df2-9247-72a2faa1a6d2 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:32.755760 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c7f358-f778-4dd0-b65e-88f2f7c0ab4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:32.756382 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c7f358-f778-4dd0-b65e-88f2f7c0ab4c None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:32.756655 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c7f358-f778-4dd0-b65e-88f2f7c0ab4c None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:32.756927 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c7f358-f778-4dd0-b65e-88f2f7c0ab4c None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:32.757567 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c7f358-f778-4dd0-b65e-88f2f7c0ab4c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:32.758974 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c7f358-f778-4dd0-b65e-88f2f7c0ab4c None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:32.848310 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 555/1134] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Oct 3 12:32:32 2022] POST /identity/v3/projects => generated 311 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:32:35.108849 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134aa845-05f1-4bbe-8a27-2f7d05fda8c9 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:35.109402 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134aa845-05f1-4bbe-8a27-2f7d05fda8c9 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:35.109402 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134aa845-05f1-4bbe-8a27-2f7d05fda8c9 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:35.110707 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 580/1135] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:32:35.116988 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2869f5-8dc8-4fd5-9a14-4ec7c2c5f68b None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:35.117369 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2869f5-8dc8-4fd5-9a14-4ec7c2c5f68b None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:35.117545 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2869f5-8dc8-4fd5-9a14-4ec7c2c5f68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:35.136294 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-3d2869f5-8dc8-4fd5-9a14-4ec7c2c5f68b None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:32:35.137470 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-3d2869f5-8dc8-4fd5-9a14-4ec7c2c5f68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:32:35.203039 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 556/1136] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:32:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:32:35.491773 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134aa845-05f1-4bbe-8a27-2f7d05fda8c9 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:35.569027 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:35.588360 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134aa845-05f1-4bbe-8a27-2f7d05fda8c9 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:35.621284 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2246553e-bfb4-448b-a24d-efb0c03a5884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:35.621900 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2246553e-bfb4-448b-a24d-efb0c03a5884 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:35.622185 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2246553e-bfb4-448b-a24d-efb0c03a5884 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:35.622460 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2246553e-bfb4-448b-a24d-efb0c03a5884 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:35.623676 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2246553e-bfb4-448b-a24d-efb0c03a5884 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:35.625087 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2246553e-bfb4-448b-a24d-efb0c03a5884 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:35.626636 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 581/1137] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:32:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:35.632315 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2869f5-8dc8-4fd5-9a14-4ec7c2c5f68b None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:35.677091 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:35.731670 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2869f5-8dc8-4fd5-9a14-4ec7c2c5f68b None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:35.764789 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a6df6-5a43-4ae6-a2be-bfd07da776ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:35.765395 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a6df6-5a43-4ae6-a2be-bfd07da776ad None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:35.765674 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a6df6-5a43-4ae6-a2be-bfd07da776ad None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:35.765947 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a6df6-5a43-4ae6-a2be-bfd07da776ad None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:35.766606 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0a6df6-5a43-4ae6-a2be-bfd07da776ad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:35.768063 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0a6df6-5a43-4ae6-a2be-bfd07da776ad None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:35.838798 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 557/1138] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Oct 3 12:32:35 2022] POST /identity/v3/users => generated 286 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:32:38.027791 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd4911-bd00-420e-be41-b8230fb3a5e4 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:38.028350 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd4911-bd00-420e-be41-b8230fb3a5e4 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:38.028350 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd4911-bd00-420e-be41-b8230fb3a5e4 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:38.029661 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 582/1139] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:32:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:32:38.034730 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf870c0-2b10-420b-9927-90a763ca4cc8 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:38.035118 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf870c0-2b10-420b-9927-90a763ca4cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:38.035272 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf870c0-2b10-420b-9927-90a763ca4cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:38.053981 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-bcf870c0-2b10-420b-9927-90a763ca4cc8 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:32:38.055145 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-bcf870c0-2b10-420b-9927-90a763ca4cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:32:38.119054 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 558/1140] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:32:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:32:38.391284 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd4911-bd00-420e-be41-b8230fb3a5e4 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:38.488192 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd4911-bd00-420e-be41-b8230fb3a5e4 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:38.520236 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b72218-635a-49e2-9cd7-ccdd15197aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:38.520845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b72218-635a-49e2-9cd7-ccdd15197aea None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:38.521123 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b72218-635a-49e2-9cd7-ccdd15197aea None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:38.521392 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b72218-635a-49e2-9cd7-ccdd15197aea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:38.531210 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b72218-635a-49e2-9cd7-ccdd15197aea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:32:38.535961 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-c5b72218-635a-49e2-9cd7-ccdd15197aea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:32:38.535961 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:32:38.535961 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:32:38.535961 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:32:38.535961 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:32:38.535961 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:32:38.535961 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:32:38.535961 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:32:38.535961 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:32:38.535961 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:32:38.535961 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:32:38.535961 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:32:38.535961 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:32:38.535961 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:32:38.535961 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:32:38.535961 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:32:38.537113 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:32:38.537113 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:32:38.537113 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:32:38.537113 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:32:38.537113 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:32:38.537113 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:32:38.537113 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:32:38.537113 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:32:38.537113 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:32:38.537113 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:32:38.537113 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:32:38.537113 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:32:38.537113 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:32:38.537113 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:32:38.537113 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:32:38.537113 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:32:38.538287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:32:38.538287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:32:38.538287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:32:38.538287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:32:38.538287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:32:38.538287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:32:38.538287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:32:38.538287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:32:38.538287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:32:38.538287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:32:38.538287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:32:38.538287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:32:38.538287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 583/1141] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 12:32:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:32:38.543214 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf870c0-2b10-420b-9927-90a763ca4cc8 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:38.572801 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:38.639116 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf870c0-2b10-420b-9927-90a763ca4cc8 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:38.672165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f4eb6a-ef01-4380-b802-cf17e32000d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:38.672702 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4eb6a-ef01-4380-b802-cf17e32000d0 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:38.672978 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4eb6a-ef01-4380-b802-cf17e32000d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:38.673248 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4eb6a-ef01-4380-b802-cf17e32000d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:38.673890 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f4eb6a-ef01-4380-b802-cf17e32000d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:38.674188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f4eb6a-ef01-4380-b802-cf17e32000d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:38.675563 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f4eb6a-ef01-4380-b802-cf17e32000d0 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:38.683802 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 559/1142] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:32:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:38.689800 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b72218-635a-49e2-9cd7-ccdd15197aea None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:38.701583 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:38.786296 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b72218-635a-49e2-9cd7-ccdd15197aea None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:38.818504 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f04cf3-724c-489f-abb1-00fcfa0f7ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:38.819359 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f04cf3-724c-489f-abb1-00fcfa0f7ef2 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:38.819644 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f04cf3-724c-489f-abb1-00fcfa0f7ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:38.819914 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f04cf3-724c-489f-abb1-00fcfa0f7ef2 None admin] PATH_INFO: `/v3/users/1bbf5cae585a4d75adecf21bd4e11010` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:38.832130 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f04cf3-724c-489f-abb1-00fcfa0f7ef2 None admin] RBAC: Authorizing `identity:get_user(user_id=1bbf5cae585a4d75adecf21bd4e11010)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:38.833542 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f04cf3-724c-489f-abb1-00fcfa0f7ef2 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:38.835671 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 584/1143] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:32:38 2022] GET /identity/v3/users/1bbf5cae585a4d75adecf21bd4e11010 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:38.841386 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f4eb6a-ef01-4380-b802-cf17e32000d0 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:38.930944 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:38.939459 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f4eb6a-ef01-4380-b802-cf17e32000d0 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:38.972684 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363ef99-50e7-4ca2-ab7d-1d474d64ae37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:38.973280 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363ef99-50e7-4ca2-ab7d-1d474d64ae37 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:38.973552 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363ef99-50e7-4ca2-ab7d-1d474d64ae37 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:38.973818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363ef99-50e7-4ca2-ab7d-1d474d64ae37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:39.038600 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b363ef99-50e7-4ca2-ab7d-1d474d64ae37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:39.040042 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b363ef99-50e7-4ca2-ab7d-1d474d64ae37 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:39.124660 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:39.133981 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 560/1144] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:32:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 12:32:39.140008 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f04cf3-724c-489f-abb1-00fcfa0f7ef2 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:39.202986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:39.236429 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f04cf3-724c-489f-abb1-00fcfa0f7ef2 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:39.269697 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87aa721-a5e0-4954-8c84-d08533fa46ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:39.270323 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87aa721-a5e0-4954-8c84-d08533fa46ee None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:39.270604 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87aa721-a5e0-4954-8c84-d08533fa46ee None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:39.270874 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87aa721-a5e0-4954-8c84-d08533fa46ee None admin] PATH_INFO: `/v3/projects/3664930e6db9499a81534c4213897a2c` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:39.272571 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87aa721-a5e0-4954-8c84-d08533fa46ee None admin] RBAC: Authorizing `identity:get_project(project_id=3664930e6db9499a81534c4213897a2c)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:39.273989 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87aa721-a5e0-4954-8c84-d08533fa46ee None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:39.275646 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 585/1145] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:32:39 2022] GET /identity/v3/projects/3664930e6db9499a81534c4213897a2c => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:39.281648 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363ef99-50e7-4ca2-ab7d-1d474d64ae37 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:39.383179 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363ef99-50e7-4ca2-ab7d-1d474d64ae37 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:39.415091 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4433b1-a054-42b4-9c31-08cecbfb7a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:39.415720 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4433b1-a054-42b4-9c31-08cecbfb7a17 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:39.416001 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4433b1-a054-42b4-9c31-08cecbfb7a17 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:39.416272 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4433b1-a054-42b4-9c31-08cecbfb7a17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:39.416986 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4433b1-a054-42b4-9c31-08cecbfb7a17 None admin] RBAC: Adding query filter params (role.id=7933c45dcdd54ef0a98018ba8a8bddaa, scope.project.id=3664930e6db9499a81534c4213897a2c, user.id=1bbf5cae585a4d75adecf21bd4e11010) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:39.417293 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4433b1-a054-42b4-9c31-08cecbfb7a17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:39.418698 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4433b1-a054-42b4-9c31-08cecbfb7a17 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:39.423881 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 561/1146] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:32:39 2022] GET /identity/v3/role_assignments?user.id=1bbf5cae585a4d75adecf21bd4e11010&scope.project.id=3664930e6db9499a81534c4213897a2c&role.id=7933c45dcdd54ef0a98018ba8a8bddaa => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:41.623371 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d99e7-5bf4-415e-93ad-b735bfb9789a None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:41.623871 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d99e7-5bf4-415e-93ad-b735bfb9789a None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:41.623871 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d99e7-5bf4-415e-93ad-b735bfb9789a None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:41.625142 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 586/1147] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:32:41.630323 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57b3d5-a288-464c-b620-f238e64869d7 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:41.630731 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57b3d5-a288-464c-b620-f238e64869d7 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:41.630867 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57b3d5-a288-464c-b620-f238e64869d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:41.649339 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-6c57b3d5-a288-464c-b620-f238e64869d7 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:32:41.650500 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-6c57b3d5-a288-464c-b620-f238e64869d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:32:41.698819 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:41.714328 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 562/1148] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:32:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:32:41.980878 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d99e7-5bf4-415e-93ad-b735bfb9789a None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:42.081608 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d99e7-5bf4-415e-93ad-b735bfb9789a None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:42.114244 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefc0f2e-5f9d-4914-9ee6-4e60374fba0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:42.114858 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc0f2e-5f9d-4914-9ee6-4e60374fba0b None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:42.115134 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc0f2e-5f9d-4914-9ee6-4e60374fba0b None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:42.115470 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc0f2e-5f9d-4914-9ee6-4e60374fba0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:42.117249 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:42.125826 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefc0f2e-5f9d-4914-9ee6-4e60374fba0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:32:42.131573 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-aefc0f2e-5f9d-4914-9ee6-4e60374fba0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:32:42.131573 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:32:42.131573 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:32:42.131573 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:32:42.131573 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:32:42.131573 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:32:42.131573 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:32:42.131573 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:32:42.131573 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:32:42.131573 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:32:42.131573 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:32:42.131573 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:32:42.131573 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:32:42.131573 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:32:42.131573 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:32:42.131573 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:32:42.133082 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:32:42.133082 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:32:42.133082 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:32:42.133082 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:32:42.133082 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:32:42.133082 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:32:42.133082 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:32:42.133082 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:32:42.133082 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:32:42.133082 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:32:42.133082 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:32:42.133082 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:32:42.133082 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:32:42.133082 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:32:42.133082 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:32:42.133082 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:32:42.134154 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:32:42.134154 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:32:42.134154 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:32:42.134154 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:32:42.134154 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:32:42.134154 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:32:42.134154 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:32:42.134154 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:32:42.134154 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:32:42.134154 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:32:42.134154 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:32:42.134154 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:32:42.134154 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 587/1149] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 12:32:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:32:42.138235 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c57b3d5-a288-464c-b620-f238e64869d7 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:42.233484 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c57b3d5-a288-464c-b620-f238e64869d7 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:42.267366 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b9ca2-096b-4963-80c8-2cce3fa3af15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:42.267972 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b9ca2-096b-4963-80c8-2cce3fa3af15 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:42.268248 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b9ca2-096b-4963-80c8-2cce3fa3af15 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:42.268514 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b9ca2-096b-4963-80c8-2cce3fa3af15 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:42.269155 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7b9ca2-096b-4963-80c8-2cce3fa3af15 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:42.269450 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7b9ca2-096b-4963-80c8-2cce3fa3af15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:42.270791 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7b9ca2-096b-4963-80c8-2cce3fa3af15 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:42.279092 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 563/1150] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:32:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:42.284840 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefc0f2e-5f9d-4914-9ee6-4e60374fba0b None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:42.384800 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefc0f2e-5f9d-4914-9ee6-4e60374fba0b None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:42.418499 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c336c-7cc4-4811-b8c8-d7fc6f557985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:42.419120 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c336c-7cc4-4811-b8c8-d7fc6f557985 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:42.419428 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c336c-7cc4-4811-b8c8-d7fc6f557985 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:42.419702 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c336c-7cc4-4811-b8c8-d7fc6f557985 None admin] PATH_INFO: `/v3/users/1bbf5cae585a4d75adecf21bd4e11010` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:42.420815 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c336c-7cc4-4811-b8c8-d7fc6f557985 None admin] RBAC: Authorizing `identity:get_user(user_id=1bbf5cae585a4d75adecf21bd4e11010)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:42.422241 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c336c-7cc4-4811-b8c8-d7fc6f557985 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:42.423845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 588/1151] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:32:42 2022] GET /identity/v3/users/1bbf5cae585a4d75adecf21bd4e11010 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:42.430442 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b9ca2-096b-4963-80c8-2cce3fa3af15 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:42.478491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:42.527100 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b9ca2-096b-4963-80c8-2cce3fa3af15 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:42.559928 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d65a2a-c636-4019-992d-2e0d704f7d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:42.560544 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d65a2a-c636-4019-992d-2e0d704f7d26 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:42.560820 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d65a2a-c636-4019-992d-2e0d704f7d26 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:42.561092 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d65a2a-c636-4019-992d-2e0d704f7d26 None admin] PATH_INFO: `/v3/projects/3664930e6db9499a81534c4213897a2c` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:42.562172 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d65a2a-c636-4019-992d-2e0d704f7d26 None admin] RBAC: Authorizing `identity:get_project(project_id=3664930e6db9499a81534c4213897a2c)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:42.563609 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d65a2a-c636-4019-992d-2e0d704f7d26 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:42.565191 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 564/1152] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:32:42 2022] GET /identity/v3/projects/3664930e6db9499a81534c4213897a2c => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:42.571875 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c336c-7cc4-4811-b8c8-d7fc6f557985 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:42.631739 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:42.669699 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c336c-7cc4-4811-b8c8-d7fc6f557985 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:42.702088 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5b14be-fc46-4808-8915-ff7b7fd11db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:42.702730 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b14be-fc46-4808-8915-ff7b7fd11db6 None admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:42.703009 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b14be-fc46-4808-8915-ff7b7fd11db6 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:42.703280 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b14be-fc46-4808-8915-ff7b7fd11db6 None admin] PATH_INFO: `/v3/projects/3664930e6db9499a81534c4213897a2c/users/1bbf5cae585a4d75adecf21bd4e11010/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:42.705145 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5b14be-fc46-4808-8915-ff7b7fd11db6 None admin] RBAC: Authorizing `identity:create_grant(project_id=3664930e6db9499a81534c4213897a2c, user_id=1bbf5cae585a4d75adecf21bd4e11010, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:42.706542 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5b14be-fc46-4808-8915-ff7b7fd11db6 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:42.786007 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 589/1153] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 12:32:42 2022] PUT /identity/v3/projects/3664930e6db9499a81534c4213897a2c/users/1bbf5cae585a4d75adecf21bd4e11010/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:32:44.918040 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98235c07-9ae9-445b-998c-387fe19e34ff None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:44.921423 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98235c07-9ae9-445b-998c-387fe19e34ff None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:44.921423 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98235c07-9ae9-445b-998c-387fe19e34ff None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:44.921423 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 565/1154] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:32:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:32:44.925067 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ebe80-d431-4649-a933-0d9c777291b6 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:44.925460 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ebe80-d431-4649-a933-0d9c777291b6 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:44.925616 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ebe80-d431-4649-a933-0d9c777291b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:44.943288 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-1b9ebe80-d431-4649-a933-0d9c777291b6 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:32:44.945015 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-1b9ebe80-d431-4649-a933-0d9c777291b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:32:45.014036 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 590/1155] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:32:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:32:45.296723 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98235c07-9ae9-445b-998c-387fe19e34ff None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:45.397940 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98235c07-9ae9-445b-998c-387fe19e34ff None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:45.413799 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:45.430943 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d154a9-39f4-4e2c-9a51-8850dd9b5c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:45.431594 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d154a9-39f4-4e2c-9a51-8850dd9b5c80 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:45.431869 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d154a9-39f4-4e2c-9a51-8850dd9b5c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:45.432138 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d154a9-39f4-4e2c-9a51-8850dd9b5c80 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:45.441786 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d154a9-39f4-4e2c-9a51-8850dd9b5c80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:32:45.447928 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-c6d154a9-39f4-4e2c-9a51-8850dd9b5c80 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:32:45.447928 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:32:45.447928 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:32:45.447928 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:32:45.447928 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:32:45.447928 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:32:45.447928 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:32:45.447928 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:32:45.447928 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:32:45.447928 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:32:45.447928 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:32:45.447928 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:32:45.447928 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:32:45.447928 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:32:45.447928 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:32:45.447928 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:32:45.449171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:32:45.449171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:32:45.449171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:32:45.449171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:32:45.449171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:32:45.449171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:32:45.449171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:32:45.449171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:32:45.449171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:32:45.449171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:32:45.449171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:32:45.449171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:32:45.449171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:32:45.449171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:32:45.449171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:32:45.449171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:32:45.450305 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:32:45.450305 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:32:45.450305 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:32:45.450305 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:32:45.450305 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:32:45.450305 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:32:45.450305 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:32:45.450305 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:32:45.450305 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:32:45.450305 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:32:45.450305 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:32:45.450305 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:32:45.450305 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 566/1156] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 12:32:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:32:45.455530 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ebe80-d431-4649-a933-0d9c777291b6 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:45.508370 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:45.555271 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ebe80-d431-4649-a933-0d9c777291b6 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:45.587971 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ac579-07aa-44aa-9d07-b8739d7fe193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:45.588560 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ac579-07aa-44aa-9d07-b8739d7fe193 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:45.588836 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ac579-07aa-44aa-9d07-b8739d7fe193 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:45.589108 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ac579-07aa-44aa-9d07-b8739d7fe193 None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:45.589755 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6ac579-07aa-44aa-9d07-b8739d7fe193 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:45.590053 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6ac579-07aa-44aa-9d07-b8739d7fe193 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:45.591413 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6ac579-07aa-44aa-9d07-b8739d7fe193 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:45.599256 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 591/1157] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:32:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:45.605777 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d154a9-39f4-4e2c-9a51-8850dd9b5c80 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:45.701781 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d154a9-39f4-4e2c-9a51-8850dd9b5c80 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:45.734707 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360742c-6e81-48c7-9c9a-4d10408112b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:45.735389 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360742c-6e81-48c7-9c9a-4d10408112b5 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:45.735626 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360742c-6e81-48c7-9c9a-4d10408112b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:45.735896 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360742c-6e81-48c7-9c9a-4d10408112b5 None admin] PATH_INFO: `/v3/users/1bbf5cae585a4d75adecf21bd4e11010` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:45.736996 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4360742c-6e81-48c7-9c9a-4d10408112b5 None admin] RBAC: Authorizing `identity:get_user(user_id=1bbf5cae585a4d75adecf21bd4e11010)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:45.738413 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4360742c-6e81-48c7-9c9a-4d10408112b5 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:45.739990 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 567/1158] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:32:45 2022] GET /identity/v3/users/1bbf5cae585a4d75adecf21bd4e11010 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:45.745622 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ac579-07aa-44aa-9d07-b8739d7fe193 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:45.841079 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ac579-07aa-44aa-9d07-b8739d7fe193 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:45.853999 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:45.872739 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb7666-f425-4dab-bda9-a37631bd027d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:45.873330 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb7666-f425-4dab-bda9-a37631bd027d None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:45.873608 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb7666-f425-4dab-bda9-a37631bd027d None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:45.873876 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb7666-f425-4dab-bda9-a37631bd027d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:45.937502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abb7666-f425-4dab-bda9-a37631bd027d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:45.938934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abb7666-f425-4dab-bda9-a37631bd027d None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:46.031967 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 592/1159] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:32:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 12:32:46.037999 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360742c-6e81-48c7-9c9a-4d10408112b5 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:46.082822 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:46.135769 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360742c-6e81-48c7-9c9a-4d10408112b5 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:46.170118 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43f743-d5ec-4de0-8e53-3fb9e0935857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:46.170742 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43f743-d5ec-4de0-8e53-3fb9e0935857 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:46.171021 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43f743-d5ec-4de0-8e53-3fb9e0935857 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:46.171295 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43f743-d5ec-4de0-8e53-3fb9e0935857 None admin] PATH_INFO: `/v3/projects/3664930e6db9499a81534c4213897a2c` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:46.172419 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43f743-d5ec-4de0-8e53-3fb9e0935857 None admin] RBAC: Authorizing `identity:get_project(project_id=3664930e6db9499a81534c4213897a2c)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:46.173846 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43f743-d5ec-4de0-8e53-3fb9e0935857 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:46.175478 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 568/1160] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:32:46 2022] GET /identity/v3/projects/3664930e6db9499a81534c4213897a2c => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:46.181713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb7666-f425-4dab-bda9-a37631bd027d None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:46.196475 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:46.280884 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb7666-f425-4dab-bda9-a37631bd027d None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:46.313975 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc894444-ccf2-456c-bb97-c48db784f6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:46.314568 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc894444-ccf2-456c-bb97-c48db784f6c4 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:46.314844 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc894444-ccf2-456c-bb97-c48db784f6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:46.315109 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc894444-ccf2-456c-bb97-c48db784f6c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:46.315858 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc894444-ccf2-456c-bb97-c48db784f6c4 None admin] RBAC: Adding query filter params (role.id=7933c45dcdd54ef0a98018ba8a8bddaa, scope.project.id=3664930e6db9499a81534c4213897a2c, user.id=1bbf5cae585a4d75adecf21bd4e11010) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:46.316163 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc894444-ccf2-456c-bb97-c48db784f6c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:46.317562 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc894444-ccf2-456c-bb97-c48db784f6c4 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:46.323220 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 593/1161] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:32:46 2022] GET /identity/v3/role_assignments?user.id=1bbf5cae585a4d75adecf21bd4e11010&scope.project.id=3664930e6db9499a81534c4213897a2c&role.id=7933c45dcdd54ef0a98018ba8a8bddaa => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:48.520902 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11774bad-1fb6-481a-9faf-86eb1f632ca7 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:48.521434 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11774bad-1fb6-481a-9faf-86eb1f632ca7 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:48.521434 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11774bad-1fb6-481a-9faf-86eb1f632ca7 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:48.523182 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 569/1162] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:32:48.528590 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5171abc-6322-4dd8-97f6-eff9b06ea405 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:48.528968 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5171abc-6322-4dd8-97f6-eff9b06ea405 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:48.529128 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5171abc-6322-4dd8-97f6-eff9b06ea405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:48.547185 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-a5171abc-6322-4dd8-97f6-eff9b06ea405 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:32:48.548373 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-a5171abc-6322-4dd8-97f6-eff9b06ea405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:32:48.611105 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 594/1163] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:32:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:32:48.878770 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11774bad-1fb6-481a-9faf-86eb1f632ca7 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:48.982231 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11774bad-1fb6-481a-9faf-86eb1f632ca7 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:48.995289 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:49.016238 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7ad2ea-4cc0-461c-b85f-817249bd1907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:49.016832 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ad2ea-4cc0-461c-b85f-817249bd1907 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:49.019676 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ad2ea-4cc0-461c-b85f-817249bd1907 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:49.019676 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ad2ea-4cc0-461c-b85f-817249bd1907 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:49.085728 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7ad2ea-4cc0-461c-b85f-817249bd1907 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:49.087177 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7ad2ea-4cc0-461c-b85f-817249bd1907 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:49.184198 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 570/1164] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:32:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 12:32:49.190143 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5171abc-6322-4dd8-97f6-eff9b06ea405 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:49.200590 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:49.291204 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5171abc-6322-4dd8-97f6-eff9b06ea405 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:49.322666 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e161c18a-6191-4ccb-9949-6ac556e5d1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:49.323279 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161c18a-6191-4ccb-9949-6ac556e5d1ce None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:49.323629 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161c18a-6191-4ccb-9949-6ac556e5d1ce None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:49.323848 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161c18a-6191-4ccb-9949-6ac556e5d1ce None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:49.330186 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e161c18a-6191-4ccb-9949-6ac556e5d1ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:49.331636 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e161c18a-6191-4ccb-9949-6ac556e5d1ce None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:49.343244 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-e161c18a-6191-4ccb-9949-6ac556e5d1ce None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 03 12:32:49.343244 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:32:49.343244 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 03 12:32:49.343244 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 03 12:32:49.343244 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 12:32:49.343244 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 12:32:49.343244 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 12:32:49.343244 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 12:32:49.343244 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Oct 03 12:32:49.343244 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:32:49.343244 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 12:32:49.343244 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:32:49.343244 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:32:49.343244 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:32:49.343244 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:32:49.343244 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:32:49.343244 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:32:49.344485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:32:49.344485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:32:49.344485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:32:49.344485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:32:49.344485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:32:49.344485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:32:49.344485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:32:49.344485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:32:49.344485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:32:49.344485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:32:49.344485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:32:49.344485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:32:49.344485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:32:49.344485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:32:49.344485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:32:49.344485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:32:49.345546 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:32:49.345546 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:32:49.345546 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:32:49.345546 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:32:49.345546 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:32:49.345546 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:32:49.345546 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:32:49.345546 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:32:49.345546 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:32:49.345546 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:32:49.345546 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 03 12:32:49.345546 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:32:49.345546 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 03 12:32:49.345546 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:32:49.345546 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 595/1165] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Oct 3 12:32:49 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:32:49.351512 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7ad2ea-4cc0-461c-b85f-817249bd1907 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:49.365184 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:49.451182 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7ad2ea-4cc0-461c-b85f-817249bd1907 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:49.483982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0367b8-0c0f-415a-b602-62d7d8f276f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:49.484576 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0367b8-0c0f-415a-b602-62d7d8f276f1 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:49.484852 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0367b8-0c0f-415a-b602-62d7d8f276f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:49.485121 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0367b8-0c0f-415a-b602-62d7d8f276f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:49.485742 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0367b8-0c0f-415a-b602-62d7d8f276f1 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:32:49.486038 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0367b8-0c0f-415a-b602-62d7d8f276f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:49.487400 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0367b8-0c0f-415a-b602-62d7d8f276f1 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:49.494224 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 571/1166] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Oct 3 12:32:49 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:32:51.629302 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d83e1-059b-445d-84bf-924b973d2c90 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:51.629829 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d83e1-059b-445d-84bf-924b973d2c90 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:51.629829 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d83e1-059b-445d-84bf-924b973d2c90 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:51.631134 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 596/1167] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:32:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:32:51.636362 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96ac85-b1d0-41f6-b1bc-89c8c4386f18 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:51.636734 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96ac85-b1d0-41f6-b1bc-89c8c4386f18 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:51.636947 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96ac85-b1d0-41f6-b1bc-89c8c4386f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:51.655176 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-5a96ac85-b1d0-41f6-b1bc-89c8c4386f18 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:32:51.656380 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-5a96ac85-b1d0-41f6-b1bc-89c8c4386f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:32:51.720339 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 572/1168] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:32:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:32:51.992857 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d83e1-059b-445d-84bf-924b973d2c90 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:52.058761 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:52.091816 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d83e1-059b-445d-84bf-924b973d2c90 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:52.125464 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c73317c-66ed-4542-809c-992867accc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:52.126076 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73317c-66ed-4542-809c-992867accc59 None admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:52.126358 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73317c-66ed-4542-809c-992867accc59 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:52.126629 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73317c-66ed-4542-809c-992867accc59 None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:52.127288 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c73317c-66ed-4542-809c-992867accc59 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:52.128661 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c73317c-66ed-4542-809c-992867accc59 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:52.225781 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 597/1169] 10.222.0.33 () {60 vars in 1138 bytes} [Mon Oct 3 12:32:51 2022] POST /identity/v3/domains => generated 267 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:32:54.375286 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b15c79-4d03-4b58-b6c3-9a90c557a4ae None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:54.375902 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b15c79-4d03-4b58-b6c3-9a90c557a4ae None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:54.375902 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b15c79-4d03-4b58-b6c3-9a90c557a4ae None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:54.377217 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 573/1170] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:32:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:32:54.383039 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0675c9e5-75ab-4b97-86d1-2962c8bdbf53 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:54.383457 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0675c9e5-75ab-4b97-86d1-2962c8bdbf53 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:54.383613 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0675c9e5-75ab-4b97-86d1-2962c8bdbf53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:54.401132 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-0675c9e5-75ab-4b97-86d1-2962c8bdbf53 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:32:54.402295 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-0675c9e5-75ab-4b97-86d1-2962c8bdbf53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:32:54.466838 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 598/1171] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:32:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:32:54.733678 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b15c79-4d03-4b58-b6c3-9a90c557a4ae None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:54.743133 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:54.837764 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b15c79-4d03-4b58-b6c3-9a90c557a4ae None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:54.872408 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b81e1c-4846-4951-8b67-f12dfc2b549d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:54.873031 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b81e1c-4846-4951-8b67-f12dfc2b549d None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:54.873312 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b81e1c-4846-4951-8b67-f12dfc2b549d None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:54.873586 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b81e1c-4846-4951-8b67-f12dfc2b549d None admin] PATH_INFO: `/v3/domains/64b9e7e53245438abc1090a9a3325a5e` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:54.884061 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b81e1c-4846-4951-8b67-f12dfc2b549d None admin] RBAC: Authorizing `identity:get_domain(domain_id=64b9e7e53245438abc1090a9a3325a5e)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:54.885511 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b81e1c-4846-4951-8b67-f12dfc2b549d None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:54.887107 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 574/1172] 10.222.0.33 () {56 vars in 1183 bytes} [Mon Oct 3 12:32:54 2022] GET /identity/v3/domains/64b9e7e53245438abc1090a9a3325a5e => generated 267 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:54.892906 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0675c9e5-75ab-4b97-86d1-2962c8bdbf53 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:54.950443 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:54.990172 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0675c9e5-75ab-4b97-86d1-2962c8bdbf53 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:55.024251 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903bbbb1-9581-4a11-8bfb-3ca90b009b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:55.028236 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903bbbb1-9581-4a11-8bfb-3ca90b009b19 None admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:55.028521 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903bbbb1-9581-4a11-8bfb-3ca90b009b19 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:55.028794 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903bbbb1-9581-4a11-8bfb-3ca90b009b19 None admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:55.030355 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903bbbb1-9581-4a11-8bfb-3ca90b009b19 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:55.032160 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903bbbb1-9581-4a11-8bfb-3ca90b009b19 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:55.128055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 599/1173] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Oct 3 12:32:54 2022] POST /identity/v3/projects => generated 361 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:32:57.348611 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17fa991-bde1-4c02-b20d-8f97b35bb108 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:57.349160 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17fa991-bde1-4c02-b20d-8f97b35bb108 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:57.349160 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17fa991-bde1-4c02-b20d-8f97b35bb108 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:57.350488 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 575/1174] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:32:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:32:57.355696 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca557a2d-b616-4436-80ca-77b483cb4d50 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:57.356094 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca557a2d-b616-4436-80ca-77b483cb4d50 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:57.356248 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca557a2d-b616-4436-80ca-77b483cb4d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:57.374591 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-ca557a2d-b616-4436-80ca-77b483cb4d50 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:32:57.375796 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-ca557a2d-b616-4436-80ca-77b483cb4d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:32:57.442457 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 600/1175] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:32:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:32:57.747664 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17fa991-bde1-4c02-b20d-8f97b35bb108 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:57.814611 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:57.852601 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17fa991-bde1-4c02-b20d-8f97b35bb108 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:57.886465 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4119f7-c960-4d55-bf42-1f4c00f78c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:57.887090 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4119f7-c960-4d55-bf42-1f4c00f78c20 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:57.887391 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4119f7-c960-4d55-bf42-1f4c00f78c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:57.887669 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4119f7-c960-4d55-bf42-1f4c00f78c20 None admin] PATH_INFO: `/v3/domains/64b9e7e53245438abc1090a9a3325a5e` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:57.888772 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4119f7-c960-4d55-bf42-1f4c00f78c20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=64b9e7e53245438abc1090a9a3325a5e)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:57.890209 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4119f7-c960-4d55-bf42-1f4c00f78c20 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:57.891800 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 576/1176] 10.222.0.33 () {56 vars in 1183 bytes} [Mon Oct 3 12:32:57 2022] GET /identity/v3/domains/64b9e7e53245438abc1090a9a3325a5e => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:32:57.898163 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca557a2d-b616-4436-80ca-77b483cb4d50 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:32:57.937791 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:32:57.993210 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca557a2d-b616-4436-80ca-77b483cb4d50 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:32:58.026795 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d6ce61-4212-4018-882b-67dde319a30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:32:58.027447 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6ce61-4212-4018-882b-67dde319a30a None admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:32:58.027727 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6ce61-4212-4018-882b-67dde319a30a None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:32:58.027997 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6ce61-4212-4018-882b-67dde319a30a None admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:32:58.028628 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d6ce61-4212-4018-882b-67dde319a30a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:32:58.030029 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d6ce61-4212-4018-882b-67dde319a30a None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:32:58.095045 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 601/1177] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Oct 3 12:32:57 2022] POST /identity/v3/users => generated 311 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:33:00.255739 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12084643-2d00-4d62-9c46-5f3fbc648db8 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:33:00.256723 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12084643-2d00-4d62-9c46-5f3fbc648db8 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:33:00.256842 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12084643-2d00-4d62-9c46-5f3fbc648db8 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:33:00.258125 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 577/1178] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:33:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:33:00.263468 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290450bf-70ee-4c7a-9fb6-ca96843a6328 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:33:00.263864 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290450bf-70ee-4c7a-9fb6-ca96843a6328 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:33:00.264018 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290450bf-70ee-4c7a-9fb6-ca96843a6328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:33:00.282471 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-290450bf-70ee-4c7a-9fb6-ca96843a6328 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:33:00.283678 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-290450bf-70ee-4c7a-9fb6-ca96843a6328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:33:00.347015 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 602/1179] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:33:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:33:00.621612 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12084643-2d00-4d62-9c46-5f3fbc648db8 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:33:00.723101 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12084643-2d00-4d62-9c46-5f3fbc648db8 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:33:00.755904 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7153e130-f57a-4d40-953b-2ac0a4501ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:33:00.757202 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153e130-f57a-4d40-953b-2ac0a4501ebb None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:33:00.757475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153e130-f57a-4d40-953b-2ac0a4501ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:33:00.757743 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153e130-f57a-4d40-953b-2ac0a4501ebb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:33:00.759557 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:33:00.767980 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7153e130-f57a-4d40-953b-2ac0a4501ebb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:33:00.773369 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-7153e130-f57a-4d40-953b-2ac0a4501ebb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:33:00.773369 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:33:00.773369 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:33:00.773369 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:33:00.773369 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:33:00.773369 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:33:00.773369 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:33:00.773369 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:33:00.773369 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:33:00.773369 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:33:00.773369 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:33:00.773369 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:33:00.773369 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:33:00.773369 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:33:00.773369 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:33:00.773369 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:33:00.782696 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:33:00.782696 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:33:00.782696 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:33:00.782696 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:33:00.782696 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:33:00.782696 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:33:00.782696 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:33:00.782696 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:33:00.782696 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:33:00.782696 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:33:00.782696 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:33:00.782696 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:33:00.782696 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:33:00.782696 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:33:00.782696 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:33:00.782696 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:33:00.783715 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:33:00.783715 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:33:00.783715 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:33:00.783715 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:33:00.783715 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:33:00.783715 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:33:00.783715 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:33:00.783715 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:33:00.783715 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:33:00.783715 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:33:00.783715 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:33:00.783715 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:33:00.783715 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 578/1180] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 12:33:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:33:00.784407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290450bf-70ee-4c7a-9fb6-ca96843a6328 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:33:00.813079 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:33:00.889541 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290450bf-70ee-4c7a-9fb6-ca96843a6328 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:33:00.922110 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855a310-a05a-42cd-8c98-5248811dba3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:33:00.922707 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855a310-a05a-42cd-8c98-5248811dba3c None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:33:00.922988 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855a310-a05a-42cd-8c98-5248811dba3c None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:33:00.923261 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855a310-a05a-42cd-8c98-5248811dba3c None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:33:00.923943 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d855a310-a05a-42cd-8c98-5248811dba3c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:33:00.924245 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d855a310-a05a-42cd-8c98-5248811dba3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:33:00.925594 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d855a310-a05a-42cd-8c98-5248811dba3c None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:33:00.933471 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 603/1181] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:33:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:33:00.939492 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7153e130-f57a-4d40-953b-2ac0a4501ebb None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:33:01.102581 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7153e130-f57a-4d40-953b-2ac0a4501ebb None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:33:01.136847 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f81f4e7-6b66-4aa7-ab0b-1bff383ae8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:33:01.137467 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81f4e7-6b66-4aa7-ab0b-1bff383ae8f0 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:33:01.137747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81f4e7-6b66-4aa7-ab0b-1bff383ae8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:33:01.138021 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81f4e7-6b66-4aa7-ab0b-1bff383ae8f0 None admin] PATH_INFO: `/v3/users/f00ff911e8164c62985f2249fd3f8a38` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:33:01.151238 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f81f4e7-6b66-4aa7-ab0b-1bff383ae8f0 None admin] RBAC: Authorizing `identity:get_user(user_id=f00ff911e8164c62985f2249fd3f8a38)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:33:01.152636 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f81f4e7-6b66-4aa7-ab0b-1bff383ae8f0 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:33:01.154217 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 579/1182] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:33:00 2022] GET /identity/v3/users/f00ff911e8164c62985f2249fd3f8a38 => generated 311 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:33:01.159977 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855a310-a05a-42cd-8c98-5248811dba3c None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:33:01.246384 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:33:01.258446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855a310-a05a-42cd-8c98-5248811dba3c None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:33:01.292193 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a70e162-de08-492a-8e7c-407029b0a96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:33:01.292795 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70e162-de08-492a-8e7c-407029b0a96a None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:33:01.293071 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70e162-de08-492a-8e7c-407029b0a96a None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:33:01.293340 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70e162-de08-492a-8e7c-407029b0a96a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:33:01.357782 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a70e162-de08-492a-8e7c-407029b0a96a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:33:01.359207 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a70e162-de08-492a-8e7c-407029b0a96a None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:33:01.442611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:33:01.454899 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 604/1183] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:33:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 12:33:01.461163 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f81f4e7-6b66-4aa7-ab0b-1bff383ae8f0 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:33:01.512618 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:33:01.558579 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f81f4e7-6b66-4aa7-ab0b-1bff383ae8f0 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:33:01.598812 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca0c937-5392-4e47-a7f4-a38ebedb6a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:33:01.599472 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0c937-5392-4e47-a7f4-a38ebedb6a30 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:33:01.599750 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0c937-5392-4e47-a7f4-a38ebedb6a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:33:01.600545 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0c937-5392-4e47-a7f4-a38ebedb6a30 None admin] PATH_INFO: `/v3/projects/3dcbf5336b3d4958bfdaac155a28cdb9` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:33:01.601654 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca0c937-5392-4e47-a7f4-a38ebedb6a30 None admin] RBAC: Authorizing `identity:get_project(project_id=3dcbf5336b3d4958bfdaac155a28cdb9)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:33:01.603080 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca0c937-5392-4e47-a7f4-a38ebedb6a30 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:33:01.604720 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 580/1184] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:33:01 2022] GET /identity/v3/projects/3dcbf5336b3d4958bfdaac155a28cdb9 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:33:01.610859 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a70e162-de08-492a-8e7c-407029b0a96a None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:33:01.705192 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a70e162-de08-492a-8e7c-407029b0a96a None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:33:01.737894 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89268b10-c5be-4c62-83c6-2f6c2aa7280a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:33:01.738486 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89268b10-c5be-4c62-83c6-2f6c2aa7280a None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:33:01.738765 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89268b10-c5be-4c62-83c6-2f6c2aa7280a None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:33:01.739035 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89268b10-c5be-4c62-83c6-2f6c2aa7280a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:33:01.739772 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89268b10-c5be-4c62-83c6-2f6c2aa7280a None admin] RBAC: Adding query filter params (role.id=7933c45dcdd54ef0a98018ba8a8bddaa, scope.project.id=3dcbf5336b3d4958bfdaac155a28cdb9, user.id=f00ff911e8164c62985f2249fd3f8a38) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:33:01.740357 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89268b10-c5be-4c62-83c6-2f6c2aa7280a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:33:01.741979 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89268b10-c5be-4c62-83c6-2f6c2aa7280a None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:33:01.747972 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 605/1185] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:33:01 2022] GET /identity/v3/role_assignments?user.id=f00ff911e8164c62985f2249fd3f8a38&scope.project.id=3dcbf5336b3d4958bfdaac155a28cdb9&role.id=7933c45dcdd54ef0a98018ba8a8bddaa => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:33:03.885280 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9975fc7-3495-4820-9058-15d09388ff71 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:33:03.885813 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9975fc7-3495-4820-9058-15d09388ff71 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:33:03.885813 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9975fc7-3495-4820-9058-15d09388ff71 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:33:03.887148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 581/1186] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:33:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:33:03.892324 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aea6a1-c0a0-464a-a436-ec8e2e9481d6 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:33:03.892760 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aea6a1-c0a0-464a-a436-ec8e2e9481d6 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:33:03.892881 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aea6a1-c0a0-464a-a436-ec8e2e9481d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:33:03.911342 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-52aea6a1-c0a0-464a-a436-ec8e2e9481d6 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:33:03.912584 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-52aea6a1-c0a0-464a-a436-ec8e2e9481d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:33:03.957762 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:33:03.977226 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 606/1187] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:33:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:33:04.246116 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9975fc7-3495-4820-9058-15d09388ff71 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:33:04.348346 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9975fc7-3495-4820-9058-15d09388ff71 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:33:04.368972 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:33:04.382601 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c6fcca-0d9c-45e0-b59a-3be8e2b115a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:33:04.383224 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c6fcca-0d9c-45e0-b59a-3be8e2b115a6 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:33:04.383532 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c6fcca-0d9c-45e0-b59a-3be8e2b115a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:33:04.383806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c6fcca-0d9c-45e0-b59a-3be8e2b115a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:33:04.393561 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c6fcca-0d9c-45e0-b59a-3be8e2b115a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:33:04.401779 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-23c6fcca-0d9c-45e0-b59a-3be8e2b115a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:33:04.401779 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:33:04.401779 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:33:04.401779 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:33:04.401779 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:33:04.401779 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:33:04.401779 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:33:04.401779 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:33:04.401779 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:33:04.401779 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:33:04.401779 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:33:04.401779 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:33:04.401779 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:33:04.401779 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:33:04.401779 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:33:04.401779 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:33:04.402908 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:33:04.402908 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:33:04.402908 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:33:04.402908 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:33:04.402908 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:33:04.402908 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:33:04.402908 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:33:04.402908 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:33:04.402908 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:33:04.402908 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:33:04.402908 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:33:04.402908 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:33:04.402908 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:33:04.402908 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:33:04.402908 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:33:04.402908 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:33:04.404011 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:33:04.404011 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:33:04.404011 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:33:04.404011 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:33:04.404011 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:33:04.404011 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:33:04.404011 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:33:04.404011 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:33:04.404011 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:33:04.404011 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:33:04.404011 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:33:04.404011 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:33:04.404011 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 582/1188] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 12:33:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:33:04.406505 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aea6a1-c0a0-464a-a436-ec8e2e9481d6 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:33:04.500872 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aea6a1-c0a0-464a-a436-ec8e2e9481d6 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:33:04.533415 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee602a83-c031-47cc-9040-026f3601e960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:33:04.534014 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee602a83-c031-47cc-9040-026f3601e960 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:33:04.534317 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee602a83-c031-47cc-9040-026f3601e960 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:33:04.534588 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee602a83-c031-47cc-9040-026f3601e960 None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:33:04.535233 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee602a83-c031-47cc-9040-026f3601e960 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:33:04.535562 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee602a83-c031-47cc-9040-026f3601e960 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:33:04.536903 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee602a83-c031-47cc-9040-026f3601e960 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:33:04.544778 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 607/1189] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:33:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:33:04.550587 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c6fcca-0d9c-45e0-b59a-3be8e2b115a6 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:33:04.646480 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c6fcca-0d9c-45e0-b59a-3be8e2b115a6 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:33:04.680147 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5a8cc-ddd5-4a0c-9e99-cf13da9b1737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:33:04.680757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5a8cc-ddd5-4a0c-9e99-cf13da9b1737 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:33:04.681033 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5a8cc-ddd5-4a0c-9e99-cf13da9b1737 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:33:04.681302 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5a8cc-ddd5-4a0c-9e99-cf13da9b1737 None admin] PATH_INFO: `/v3/users/f00ff911e8164c62985f2249fd3f8a38` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:33:04.682383 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d5a8cc-ddd5-4a0c-9e99-cf13da9b1737 None admin] RBAC: Authorizing `identity:get_user(user_id=f00ff911e8164c62985f2249fd3f8a38)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:33:04.683838 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d5a8cc-ddd5-4a0c-9e99-cf13da9b1737 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:33:04.685377 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 583/1190] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:33:04 2022] GET /identity/v3/users/f00ff911e8164c62985f2249fd3f8a38 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:33:04.691045 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee602a83-c031-47cc-9040-026f3601e960 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:33:04.736510 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:33:04.790014 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee602a83-c031-47cc-9040-026f3601e960 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:33:04.827866 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3e7d8f-8c19-4d2b-b05b-cd3c201bf5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:33:04.828496 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e7d8f-8c19-4d2b-b05b-cd3c201bf5c7 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:33:04.828778 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e7d8f-8c19-4d2b-b05b-cd3c201bf5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:33:04.829051 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e7d8f-8c19-4d2b-b05b-cd3c201bf5c7 None admin] PATH_INFO: `/v3/projects/3dcbf5336b3d4958bfdaac155a28cdb9` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:33:04.830161 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3e7d8f-8c19-4d2b-b05b-cd3c201bf5c7 None admin] RBAC: Authorizing `identity:get_project(project_id=3dcbf5336b3d4958bfdaac155a28cdb9)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:33:04.831602 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3e7d8f-8c19-4d2b-b05b-cd3c201bf5c7 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:33:04.833224 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 608/1191] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:33:04 2022] GET /identity/v3/projects/3dcbf5336b3d4958bfdaac155a28cdb9 => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:33:04.839401 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5a8cc-ddd5-4a0c-9e99-cf13da9b1737 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:33:04.887552 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:33:04.935835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5a8cc-ddd5-4a0c-9e99-cf13da9b1737 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:33:04.968021 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f714b555-955f-4999-84f3-69ac8f329b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:33:04.968670 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714b555-955f-4999-84f3-69ac8f329b89 None admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:33:04.968947 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714b555-955f-4999-84f3-69ac8f329b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:33:04.969219 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714b555-955f-4999-84f3-69ac8f329b89 None admin] PATH_INFO: `/v3/projects/3dcbf5336b3d4958bfdaac155a28cdb9/users/f00ff911e8164c62985f2249fd3f8a38/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:33:04.971075 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f714b555-955f-4999-84f3-69ac8f329b89 None admin] RBAC: Authorizing `identity:create_grant(project_id=3dcbf5336b3d4958bfdaac155a28cdb9, user_id=f00ff911e8164c62985f2249fd3f8a38, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:33:04.972951 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f714b555-955f-4999-84f3-69ac8f329b89 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:33:05.070718 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 584/1192] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 12:33:04 2022] PUT /identity/v3/projects/3dcbf5336b3d4958bfdaac155a28cdb9/users/f00ff911e8164c62985f2249fd3f8a38/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:33:07.177388 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef99317-7e6c-4b19-9b83-ddee971efc25 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:33:07.177922 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef99317-7e6c-4b19-9b83-ddee971efc25 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:33:07.177922 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef99317-7e6c-4b19-9b83-ddee971efc25 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:33:07.179830 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 609/1193] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:33:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:33:07.185124 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39769358-3e7c-4e8f-a33b-8d5582d4b44a None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:33:07.185539 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39769358-3e7c-4e8f-a33b-8d5582d4b44a None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:33:07.185671 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39769358-3e7c-4e8f-a33b-8d5582d4b44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:33:07.204117 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-39769358-3e7c-4e8f-a33b-8d5582d4b44a None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:33:07.205295 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-39769358-3e7c-4e8f-a33b-8d5582d4b44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:33:07.269710 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 585/1194] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:33:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:33:07.540555 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef99317-7e6c-4b19-9b83-ddee971efc25 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:33:07.639884 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef99317-7e6c-4b19-9b83-ddee971efc25 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:33:07.652439 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:33:07.672593 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65883037-06b0-4f6e-be09-c0224f77e843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:33:07.673216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65883037-06b0-4f6e-be09-c0224f77e843 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:33:07.673496 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65883037-06b0-4f6e-be09-c0224f77e843 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:33:07.673765 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65883037-06b0-4f6e-be09-c0224f77e843 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:33:07.684597 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65883037-06b0-4f6e-be09-c0224f77e843 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:33:07.689393 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-65883037-06b0-4f6e-be09-c0224f77e843 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:33:07.689393 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:33:07.689393 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:33:07.689393 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:33:07.689393 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:33:07.689393 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:33:07.689393 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:33:07.689393 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:33:07.689393 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:33:07.689393 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:33:07.689393 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:33:07.689393 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:33:07.689393 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:33:07.689393 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:33:07.689393 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:33:07.689393 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:33:07.690504 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:33:07.690504 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:33:07.690504 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:33:07.690504 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:33:07.690504 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:33:07.690504 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:33:07.690504 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:33:07.690504 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:33:07.690504 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:33:07.690504 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:33:07.690504 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:33:07.690504 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:33:07.690504 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:33:07.690504 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:33:07.690504 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:33:07.690504 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:33:07.691752 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:33:07.691752 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:33:07.691752 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:33:07.691752 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:33:07.691752 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:33:07.691752 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:33:07.691752 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:33:07.691752 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:33:07.691752 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:33:07.691752 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:33:07.691752 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:33:07.691752 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:33:07.691752 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 610/1195] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 12:33:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:33:07.696681 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39769358-3e7c-4e8f-a33b-8d5582d4b44a None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:33:07.735631 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:33:07.793728 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39769358-3e7c-4e8f-a33b-8d5582d4b44a None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:33:07.825492 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df49258-bf4d-4eb5-9959-3d525d9f462b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:33:07.826094 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df49258-bf4d-4eb5-9959-3d525d9f462b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:33:07.826369 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df49258-bf4d-4eb5-9959-3d525d9f462b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:33:07.826637 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df49258-bf4d-4eb5-9959-3d525d9f462b None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:33:07.827279 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df49258-bf4d-4eb5-9959-3d525d9f462b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:33:07.827615 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df49258-bf4d-4eb5-9959-3d525d9f462b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:33:07.828959 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df49258-bf4d-4eb5-9959-3d525d9f462b None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:33:07.837302 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 586/1196] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:33:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:33:07.843172 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65883037-06b0-4f6e-be09-c0224f77e843 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:33:07.938082 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65883037-06b0-4f6e-be09-c0224f77e843 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:33:07.970068 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a29e18-f09f-4cbc-811b-c2c3e71e06fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:33:07.970669 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a29e18-f09f-4cbc-811b-c2c3e71e06fe None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:33:07.970943 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a29e18-f09f-4cbc-811b-c2c3e71e06fe None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:33:07.971213 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a29e18-f09f-4cbc-811b-c2c3e71e06fe None admin] PATH_INFO: `/v3/users/f00ff911e8164c62985f2249fd3f8a38` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:33:07.972585 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a29e18-f09f-4cbc-811b-c2c3e71e06fe None admin] RBAC: Authorizing `identity:get_user(user_id=f00ff911e8164c62985f2249fd3f8a38)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:33:07.973982 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a29e18-f09f-4cbc-811b-c2c3e71e06fe None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:33:07.975796 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 611/1197] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 12:33:07 2022] GET /identity/v3/users/f00ff911e8164c62985f2249fd3f8a38 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:33:07.981765 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df49258-bf4d-4eb5-9959-3d525d9f462b None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:33:08.077078 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:33:08.079889 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df49258-bf4d-4eb5-9959-3d525d9f462b None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:33:08.112446 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b1c19-26a9-4559-b3ed-238437c0be93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:33:08.113040 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b1c19-26a9-4559-b3ed-238437c0be93 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:33:08.113311 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b1c19-26a9-4559-b3ed-238437c0be93 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:33:08.113577 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b1c19-26a9-4559-b3ed-238437c0be93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:33:08.177294 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9b1c19-26a9-4559-b3ed-238437c0be93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:33:08.178856 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9b1c19-26a9-4559-b3ed-238437c0be93 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:33:08.269208 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:33:08.272868 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 587/1198] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:33:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 12:33:08.278751 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a29e18-f09f-4cbc-811b-c2c3e71e06fe None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:33:08.319927 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:33:08.375459 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a29e18-f09f-4cbc-811b-c2c3e71e06fe None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:33:08.406904 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7d90a-ec9c-4278-9ad2-80b2f47c42fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:33:08.407564 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7d90a-ec9c-4278-9ad2-80b2f47c42fd None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:33:08.407846 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7d90a-ec9c-4278-9ad2-80b2f47c42fd None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:33:08.408118 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7d90a-ec9c-4278-9ad2-80b2f47c42fd None admin] PATH_INFO: `/v3/projects/3dcbf5336b3d4958bfdaac155a28cdb9` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:33:08.409215 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd7d90a-ec9c-4278-9ad2-80b2f47c42fd None admin] RBAC: Authorizing `identity:get_project(project_id=3dcbf5336b3d4958bfdaac155a28cdb9)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:33:08.410634 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd7d90a-ec9c-4278-9ad2-80b2f47c42fd None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:33:08.412290 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 612/1199] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 12:33:08 2022] GET /identity/v3/projects/3dcbf5336b3d4958bfdaac155a28cdb9 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:33:08.419260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b1c19-26a9-4559-b3ed-238437c0be93 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:33:08.514764 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b1c19-26a9-4559-b3ed-238437c0be93 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:33:08.547812 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f6f05d-fb32-4877-9a25-f33f528fc49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:33:08.548405 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6f05d-fb32-4877-9a25-f33f528fc49b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:33:08.548682 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6f05d-fb32-4877-9a25-f33f528fc49b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:33:08.548955 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6f05d-fb32-4877-9a25-f33f528fc49b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:33:08.549660 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f6f05d-fb32-4877-9a25-f33f528fc49b None admin] RBAC: Adding query filter params (role.id=7933c45dcdd54ef0a98018ba8a8bddaa, scope.project.id=3dcbf5336b3d4958bfdaac155a28cdb9, user.id=f00ff911e8164c62985f2249fd3f8a38) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:33:08.549964 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f6f05d-fb32-4877-9a25-f33f528fc49b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:33:08.551399 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f6f05d-fb32-4877-9a25-f33f528fc49b None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:33:08.557683 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 588/1200] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:33:08 2022] GET /identity/v3/role_assignments?user.id=f00ff911e8164c62985f2249fd3f8a38&scope.project.id=3dcbf5336b3d4958bfdaac155a28cdb9&role.id=7933c45dcdd54ef0a98018ba8a8bddaa => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:33:18.402684 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=46 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:33:18.555280 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=68 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:22.073536 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a9e54-0c8b-4cf6-b046-96268cdbed83 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:22.074144 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a9e54-0c8b-4cf6-b046-96268cdbed83 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:22.074545 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a9e54-0c8b-4cf6-b046-96268cdbed83 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:22.076164 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 613/1201] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:34:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:34:22.083101 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4548be4d-76b6-4fb2-88b2-a3193ff15dde None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:22.083658 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4548be4d-76b6-4fb2-88b2-a3193ff15dde None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:22.084883 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4548be4d-76b6-4fb2-88b2-a3193ff15dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:22.111892 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-4548be4d-76b6-4fb2-88b2-a3193ff15dde None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:34:22.114228 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4548be4d-76b6-4fb2-88b2-a3193ff15dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:34:22.191092 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 589/1202] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:34:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:34:22.465486 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a9e54-0c8b-4cf6-b046-96268cdbed83 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:22.567929 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a9e54-0c8b-4cf6-b046-96268cdbed83 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:22.602916 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd96e76-4ef2-4294-a306-adfe282a0c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:22.603564 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd96e76-4ef2-4294-a306-adfe282a0c0b None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:22.603845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd96e76-4ef2-4294-a306-adfe282a0c0b None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:22.604111 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd96e76-4ef2-4294-a306-adfe282a0c0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:22.613928 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd96e76-4ef2-4294-a306-adfe282a0c0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:22.615362 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd96e76-4ef2-4294-a306-adfe282a0c0b None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:22.630211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-dbd96e76-4ef2-4294-a306-adfe282a0c0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:34:22.630211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:34:22.630211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:34:22.630211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:34:22.630211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:34:22.630211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:34:22.630211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:34:22.630211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:34:22.630211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:34:22.630211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:34:22.630211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:34:22.630211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:34:22.630211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:34:22.630211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:34:22.630211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:34:22.630211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:34:22.631444 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:34:22.631444 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:34:22.631444 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:34:22.631444 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:34:22.631444 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:34:22.631444 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:34:22.631444 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:34:22.631444 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:34:22.631444 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:34:22.631444 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:34:22.631444 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:34:22.631444 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:34:22.631444 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:34:22.631444 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:34:22.631444 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:34:22.631444 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:34:22.631444 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:34:22.632788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:34:22.632788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:34:22.632788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:34:22.632788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 614/1203] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:34:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:34:22.637600 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4548be4d-76b6-4fb2-88b2-a3193ff15dde None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:22.736285 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4548be4d-76b6-4fb2-88b2-a3193ff15dde None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:22.764946 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:22.771451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f195aaaa-78aa-41a4-b324-88138d22d121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:22.771451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195aaaa-78aa-41a4-b324-88138d22d121 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:22.771451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195aaaa-78aa-41a4-b324-88138d22d121 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:22.771747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195aaaa-78aa-41a4-b324-88138d22d121 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:22.772341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f195aaaa-78aa-41a4-b324-88138d22d121 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:34:22.772635 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f195aaaa-78aa-41a4-b324-88138d22d121 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:22.773975 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f195aaaa-78aa-41a4-b324-88138d22d121 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:22.784979 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 590/1204] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:34:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:34:22.790615 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd96e76-4ef2-4294-a306-adfe282a0c0b None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:22.843391 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:22.888608 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd96e76-4ef2-4294-a306-adfe282a0c0b None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:22.921166 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903ae9d-48ec-43df-ab6c-4d0278b29b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:22.921756 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903ae9d-48ec-43df-ab6c-4d0278b29b2e None admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:22.922038 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903ae9d-48ec-43df-ab6c-4d0278b29b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:22.922308 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903ae9d-48ec-43df-ab6c-4d0278b29b2e None admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:22.922942 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e903ae9d-48ec-43df-ab6c-4d0278b29b2e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:22.924516 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e903ae9d-48ec-43df-ab6c-4d0278b29b2e None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:23.017152 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 615/1205] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Oct 3 12:34:22 2022] POST /identity/v3/users => generated 251 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:34:25.190215 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1afc3-fc1b-4958-adbf-ff0a51726276 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:25.190786 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1afc3-fc1b-4958-adbf-ff0a51726276 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:25.190786 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1afc3-fc1b-4958-adbf-ff0a51726276 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:25.192142 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 591/1206] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:34:25.197422 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550972aa-66f3-41d3-bb67-f98d21f3ea4f None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:25.197874 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550972aa-66f3-41d3-bb67-f98d21f3ea4f None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:25.198025 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550972aa-66f3-41d3-bb67-f98d21f3ea4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:25.217669 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-550972aa-66f3-41d3-bb67-f98d21f3ea4f None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:34:25.218900 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-550972aa-66f3-41d3-bb67-f98d21f3ea4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:34:25.285132 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 616/1207] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:34:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:34:25.556471 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a1afc3-fc1b-4958-adbf-ff0a51726276 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:25.659215 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a1afc3-fc1b-4958-adbf-ff0a51726276 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:25.692441 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef4ad1-9ebb-427f-bc1c-4b721e1f3c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:25.694577 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef4ad1-9ebb-427f-bc1c-4b721e1f3c99 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:25.694577 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef4ad1-9ebb-427f-bc1c-4b721e1f3c99 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:25.694577 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef4ad1-9ebb-427f-bc1c-4b721e1f3c99 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:25.703409 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef4ad1-9ebb-427f-bc1c-4b721e1f3c99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:34:25.708122 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-8aef4ad1-9ebb-427f-bc1c-4b721e1f3c99 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:34:25.708122 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:34:25.708122 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:34:25.708122 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:34:25.708122 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:34:25.708122 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:34:25.708122 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:34:25.708122 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:34:25.708122 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:34:25.708122 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:34:25.708122 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:34:25.708122 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:34:25.708122 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:34:25.708122 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:34:25.708122 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:34:25.708122 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:34:25.709210 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:34:25.709210 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:34:25.709210 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:34:25.709210 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:34:25.709210 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:34:25.709210 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:34:25.709210 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:34:25.709210 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:34:25.709210 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:34:25.709210 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:34:25.709210 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:34:25.709210 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:34:25.709210 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:34:25.709210 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:34:25.709210 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:34:25.709210 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:34:25.710338 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:34:25.710338 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:34:25.710338 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:34:25.710338 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:34:25.710338 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:34:25.710338 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:34:25.710338 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:34:25.710338 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:34:25.710338 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:34:25.710338 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:34:25.710338 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:34:25.710338 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:34:25.710338 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 592/1208] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 12:34:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:34:25.715381 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550972aa-66f3-41d3-bb67-f98d21f3ea4f None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:25.754528 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:25.810928 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550972aa-66f3-41d3-bb67-f98d21f3ea4f None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:25.843116 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99aea9-9e77-4532-8e82-8e6eb8fdab1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:25.843739 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99aea9-9e77-4532-8e82-8e6eb8fdab1c None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:25.844018 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99aea9-9e77-4532-8e82-8e6eb8fdab1c None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:25.844285 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99aea9-9e77-4532-8e82-8e6eb8fdab1c None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:25.844929 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b99aea9-9e77-4532-8e82-8e6eb8fdab1c None admin] RBAC: Adding query filter params (name=service) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:34:25.845227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b99aea9-9e77-4532-8e82-8e6eb8fdab1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:25.846566 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b99aea9-9e77-4532-8e82-8e6eb8fdab1c None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:25.854285 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 617/1209] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 12:34:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:34:25.860376 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef4ad1-9ebb-427f-bc1c-4b721e1f3c99 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:25.910884 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:25.963640 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef4ad1-9ebb-427f-bc1c-4b721e1f3c99 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:25.995785 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1644835-73b1-4022-a35e-2182a9449596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:25.996417 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1644835-73b1-4022-a35e-2182a9449596 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:25.996698 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1644835-73b1-4022-a35e-2182a9449596 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:25.996971 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1644835-73b1-4022-a35e-2182a9449596 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:26.010355 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1644835-73b1-4022-a35e-2182a9449596 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:26.013744 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1644835-73b1-4022-a35e-2182a9449596 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:26.030342 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-b1644835-73b1-4022-a35e-2182a9449596 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:34:26.030342 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:34:26.030342 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:34:26.030342 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:34:26.030342 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:34:26.030342 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:34:26.030342 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:34:26.030342 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:34:26.030342 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:34:26.030342 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:34:26.030342 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:34:26.030342 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:34:26.030342 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:34:26.030342 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:34:26.030342 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:34:26.030342 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:34:26.031376 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:34:26.031376 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:34:26.031376 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:34:26.031376 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:34:26.031376 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:34:26.031376 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:34:26.031376 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:34:26.031376 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:34:26.031376 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:34:26.031376 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:34:26.031376 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:34:26.031376 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:34:26.031376 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:34:26.031376 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:34:26.031376 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:34:26.031376 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:34:26.031376 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:34:26.032354 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:34:26.032354 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:34:26.032354 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:34:26.032354 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 593/1210] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:34:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:34:26.037712 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99aea9-9e77-4532-8e82-8e6eb8fdab1c None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:26.136807 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:26.140183 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99aea9-9e77-4532-8e82-8e6eb8fdab1c None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:26.173171 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a273a877-9c07-41c4-92b6-78576ed84b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:26.173764 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273a877-9c07-41c4-92b6-78576ed84b87 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:26.174048 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273a877-9c07-41c4-92b6-78576ed84b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:26.174326 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273a877-9c07-41c4-92b6-78576ed84b87 None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:26.174987 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a273a877-9c07-41c4-92b6-78576ed84b87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:34:26.175306 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a273a877-9c07-41c4-92b6-78576ed84b87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:26.176674 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a273a877-9c07-41c4-92b6-78576ed84b87 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:26.187919 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 618/1211] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:34:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:34:26.193740 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1644835-73b1-4022-a35e-2182a9449596 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:26.295143 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1644835-73b1-4022-a35e-2182a9449596 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:26.296676 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:26.329221 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b3fa1f-dc3c-4bcb-af40-b9ff8d4b613a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:26.329852 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3fa1f-dc3c-4bcb-af40-b9ff8d4b613a None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:26.330128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3fa1f-dc3c-4bcb-af40-b9ff8d4b613a None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:26.330399 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3fa1f-dc3c-4bcb-af40-b9ff8d4b613a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:26.337216 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b3fa1f-dc3c-4bcb-af40-b9ff8d4b613a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:26.338635 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b3fa1f-dc3c-4bcb-af40-b9ff8d4b613a None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:26.350460 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-73b3fa1f-dc3c-4bcb-af40-b9ff8d4b613a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 03 12:34:26.350460 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:34:26.350460 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:34:26.350460 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:34:26.350460 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:34:26.350460 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:34:26.350460 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:34:26.350460 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:34:26.350460 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:34:26.350460 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:34:26.350460 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:34:26.350460 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:34:26.350460 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 12:34:26.350460 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 12:34:26.350460 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 12:34:26.350460 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 12:34:26.351619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:34:26.351619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:34:26.351619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 12:34:26.351619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:34:26.351619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 12:34:26.351619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:34:26.351619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:34:26.351619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:34:26.351619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:34:26.351619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:34:26.351619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:34:26.351619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:34:26.351619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:34:26.351619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:34:26.351619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:34:26.351619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:34:26.351619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:34:26.351619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:34:26.352619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:34:26.352619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:34:26.352619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 12:34:26.352619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 12:34:26.352619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 12:34:26.352619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 12:34:26.352619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 12:34:26.352619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 12:34:26.352619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 03 12:34:26.352619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:34:26.352619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 594/1212] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:34:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:34:26.358094 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a273a877-9c07-41c4-92b6-78576ed84b87 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:26.454775 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a273a877-9c07-41c4-92b6-78576ed84b87 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:26.488091 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c77103-aab8-4b96-8c01-a0cb67384d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:26.488695 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c77103-aab8-4b96-8c01-a0cb67384d6a None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:26.488973 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c77103-aab8-4b96-8c01-a0cb67384d6a None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:26.489243 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c77103-aab8-4b96-8c01-a0cb67384d6a None admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:26.489958 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c77103-aab8-4b96-8c01-a0cb67384d6a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:34:26.490263 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c77103-aab8-4b96-8c01-a0cb67384d6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:26.491703 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c77103-aab8-4b96-8c01-a0cb67384d6a None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:26.509342 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 619/1213] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Oct 3 12:34:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:34:26.515139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b3fa1f-dc3c-4bcb-af40-b9ff8d4b613a None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:26.613855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b3fa1f-dc3c-4bcb-af40-b9ff8d4b613a None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:26.647712 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e944da-6668-4390-8749-84af10366e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:26.648314 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e944da-6668-4390-8749-84af10366e3c None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:26.648586 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e944da-6668-4390-8749-84af10366e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:26.648857 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e944da-6668-4390-8749-84af10366e3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:26.679620 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:26.715739 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e944da-6668-4390-8749-84af10366e3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:26.718037 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e944da-6668-4390-8749-84af10366e3c None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:26.815846 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 595/1214] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:34:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 12:34:26.821764 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c77103-aab8-4b96-8c01-a0cb67384d6a None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:26.842845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:26.923563 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c77103-aab8-4b96-8c01-a0cb67384d6a None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:26.956482 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c73a0-e628-487a-821c-f0ec7153009a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:26.957109 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c73a0-e628-487a-821c-f0ec7153009a None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:26.957645 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c73a0-e628-487a-821c-f0ec7153009a None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:26.957749 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c73a0-e628-487a-821c-f0ec7153009a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:26.968141 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38c73a0-e628-487a-821c-f0ec7153009a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:26.969583 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38c73a0-e628-487a-821c-f0ec7153009a None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:26.984277 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-d38c73a0-e628-487a-821c-f0ec7153009a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:34:26.984277 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:34:26.984277 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:34:26.984277 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:34:26.984277 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:34:26.984277 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:34:26.984277 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:34:26.984277 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:34:26.984277 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:34:26.984277 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:34:26.984277 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:34:26.984277 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:34:26.984277 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:34:26.984277 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:34:26.984277 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:34:26.984277 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:34:26.985379 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:34:26.985379 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:34:26.985379 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:34:26.985379 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:34:26.985379 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:34:26.985379 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:34:26.985379 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:34:26.985379 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:34:26.985379 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:34:26.985379 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:34:26.985379 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:34:26.985379 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:34:26.985379 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:34:26.985379 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:34:26.985379 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:34:26.985379 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:34:26.985379 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:34:26.986304 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:34:26.986304 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:34:26.986304 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:34:26.986304 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 620/1215] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:34:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:34:26.991144 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e944da-6668-4390-8749-84af10366e3c None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:27.052010 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:27.089858 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e944da-6668-4390-8749-84af10366e3c None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:27.124741 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b11a07-514d-45f1-928a-96dcc6fade3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:27.125353 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b11a07-514d-45f1-928a-96dcc6fade3c None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:27.125633 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b11a07-514d-45f1-928a-96dcc6fade3c None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:27.125907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b11a07-514d-45f1-928a-96dcc6fade3c None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:27.126539 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b11a07-514d-45f1-928a-96dcc6fade3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:34:27.126841 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b11a07-514d-45f1-928a-96dcc6fade3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:27.128225 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b11a07-514d-45f1-928a-96dcc6fade3c None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:27.140422 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 596/1216] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:34:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:34:27.150338 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c73a0-e628-487a-821c-f0ec7153009a None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:27.214223 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:27.244058 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c73a0-e628-487a-821c-f0ec7153009a None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:27.276963 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e45520a-588c-42f3-9b5b-aa06cde0fb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:27.277598 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45520a-588c-42f3-9b5b-aa06cde0fb6b None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:27.277875 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45520a-588c-42f3-9b5b-aa06cde0fb6b None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:27.278147 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45520a-588c-42f3-9b5b-aa06cde0fb6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:27.284911 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e45520a-588c-42f3-9b5b-aa06cde0fb6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:27.288127 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e45520a-588c-42f3-9b5b-aa06cde0fb6b None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:27.297994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-1e45520a-588c-42f3-9b5b-aa06cde0fb6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:34:27.297994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:34:27.297994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:34:27.297994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:34:27.297994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:34:27.297994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:34:27.297994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:34:27.297994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:34:27.297994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:34:27.297994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:34:27.297994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:34:27.297994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:34:27.297994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 12:34:27.297994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 12:34:27.297994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 12:34:27.299067 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 12:34:27.299067 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:34:27.299067 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:34:27.299067 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:34:27.299067 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:34:27.299067 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:34:27.299067 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:34:27.299067 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:34:27.299067 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:34:27.299067 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:34:27.299067 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:34:27.299067 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:34:27.299067 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:34:27.299067 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:34:27.299067 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:34:27.299067 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:34:27.299067 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:34:27.300064 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 12:34:27.300064 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 12:34:27.300064 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 12:34:27.300064 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 12:34:27.300064 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 12:34:27.300064 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 12:34:27.300064 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:34:27.300064 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:34:27.300064 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 621/1217] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:34:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:34:27.305150 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b11a07-514d-45f1-928a-96dcc6fade3c None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:27.418090 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b11a07-514d-45f1-928a-96dcc6fade3c None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:27.440559 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:27.452184 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de42c498-3d41-40dd-978d-deaceabd08c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:27.452784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42c498-3d41-40dd-978d-deaceabd08c1 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:27.453058 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42c498-3d41-40dd-978d-deaceabd08c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:27.453323 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42c498-3d41-40dd-978d-deaceabd08c1 None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:27.453990 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de42c498-3d41-40dd-978d-deaceabd08c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:34:27.454286 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de42c498-3d41-40dd-978d-deaceabd08c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:27.455779 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de42c498-3d41-40dd-978d-deaceabd08c1 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:27.467193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 597/1218] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 12:34:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:34:27.473340 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e45520a-588c-42f3-9b5b-aa06cde0fb6b None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:27.572236 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e45520a-588c-42f3-9b5b-aa06cde0fb6b None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:27.599633 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:27.605699 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a232c2-fba5-406d-a09c-595e16201135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:27.606298 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a232c2-fba5-406d-a09c-595e16201135 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:27.606577 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a232c2-fba5-406d-a09c-595e16201135 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:27.607605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a232c2-fba5-406d-a09c-595e16201135 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:27.608317 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a232c2-fba5-406d-a09c-595e16201135 None admin] RBAC: Adding query filter params (role.id=ff6b2d4413644db39a2bd8576afc6ccb, scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7, user.id=25eef64e92e9422ea62508c3eb0a197e) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:34:27.608640 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a232c2-fba5-406d-a09c-595e16201135 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:27.610033 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a232c2-fba5-406d-a09c-595e16201135 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:27.615213 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 622/1219] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:34:27 2022] GET /identity/v3/role_assignments?user.id=25eef64e92e9422ea62508c3eb0a197e&scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7&role.id=ff6b2d4413644db39a2bd8576afc6ccb => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:34:29.823277 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ba140-0753-44f3-b8b4-cba195c250ba None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:29.823858 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ba140-0753-44f3-b8b4-cba195c250ba None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:29.823951 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ba140-0753-44f3-b8b4-cba195c250ba None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:29.825295 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 598/1220] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:34:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:34:29.830502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97a93e-7ffb-4e82-855f-ddb9863ddd7e None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:29.830935 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97a93e-7ffb-4e82-855f-ddb9863ddd7e None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:29.831029 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97a93e-7ffb-4e82-855f-ddb9863ddd7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:29.848921 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-9f97a93e-7ffb-4e82-855f-ddb9863ddd7e None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:34:29.850388 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-9f97a93e-7ffb-4e82-855f-ddb9863ddd7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:34:29.914972 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 623/1221] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:34:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:34:30.195917 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ba140-0753-44f3-b8b4-cba195c250ba None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:30.298649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ba140-0753-44f3-b8b4-cba195c250ba None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:30.332340 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397769eb-1fb2-4af0-a2c5-0ad75a53898c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:30.333003 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397769eb-1fb2-4af0-a2c5-0ad75a53898c None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:30.333286 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397769eb-1fb2-4af0-a2c5-0ad75a53898c None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:30.333560 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397769eb-1fb2-4af0-a2c5-0ad75a53898c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:30.344817 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397769eb-1fb2-4af0-a2c5-0ad75a53898c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:34:30.349617 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-397769eb-1fb2-4af0-a2c5-0ad75a53898c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:34:30.349617 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:34:30.349617 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:34:30.349617 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:34:30.349617 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:34:30.349617 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:34:30.349617 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:34:30.349617 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:34:30.349617 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:34:30.349617 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:34:30.349617 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:34:30.349617 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:34:30.349617 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:34:30.349617 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:34:30.349617 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:34:30.349617 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:34:30.351013 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:34:30.351013 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:34:30.351013 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:34:30.351013 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:34:30.351013 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:34:30.351013 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:34:30.351013 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:34:30.351013 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:34:30.351013 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:34:30.351013 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:34:30.351013 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:34:30.351013 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:34:30.351013 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:34:30.351013 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:34:30.351013 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:34:30.351013 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:34:30.352130 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:34:30.352130 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:34:30.352130 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:34:30.352130 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:34:30.352130 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:34:30.352130 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:34:30.352130 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:34:30.352130 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:34:30.352130 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:34:30.352130 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:34:30.352130 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:34:30.352130 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:34:30.352130 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 599/1222] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 12:34:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:34:30.357679 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f97a93e-7ffb-4e82-855f-ddb9863ddd7e None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:30.457032 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f97a93e-7ffb-4e82-855f-ddb9863ddd7e None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:30.480515 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:30.491982 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f12b8-010a-43b1-8c84-c603a6a09e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:30.492593 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f12b8-010a-43b1-8c84-c603a6a09e0b None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:30.492877 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f12b8-010a-43b1-8c84-c603a6a09e0b None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:30.493152 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f12b8-010a-43b1-8c84-c603a6a09e0b None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:30.493851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5f12b8-010a-43b1-8c84-c603a6a09e0b None admin] RBAC: Adding query filter params (name=service) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:34:30.494159 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5f12b8-010a-43b1-8c84-c603a6a09e0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:30.495539 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5f12b8-010a-43b1-8c84-c603a6a09e0b None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:30.503558 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 624/1223] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 12:34:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:34:30.509432 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397769eb-1fb2-4af0-a2c5-0ad75a53898c None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:30.553341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:30.609738 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397769eb-1fb2-4af0-a2c5-0ad75a53898c None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:30.643827 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8174c7-ae39-4fc0-914c-3d975bc56744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:30.644444 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8174c7-ae39-4fc0-914c-3d975bc56744 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:30.644719 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8174c7-ae39-4fc0-914c-3d975bc56744 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:30.644985 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8174c7-ae39-4fc0-914c-3d975bc56744 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:30.655084 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8174c7-ae39-4fc0-914c-3d975bc56744 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:30.656580 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8174c7-ae39-4fc0-914c-3d975bc56744 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:30.670806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-ad8174c7-ae39-4fc0-914c-3d975bc56744 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:34:30.670806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:34:30.670806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:34:30.670806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:34:30.670806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:34:30.670806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:34:30.670806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:34:30.670806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:34:30.670806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:34:30.670806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:34:30.670806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:34:30.670806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:34:30.670806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:34:30.670806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:34:30.670806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:34:30.670806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:34:30.671994 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:34:30.671994 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:34:30.671994 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:34:30.671994 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:34:30.671994 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:34:30.671994 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:34:30.671994 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:34:30.671994 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:34:30.671994 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:34:30.671994 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:34:30.671994 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:34:30.671994 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:34:30.671994 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:34:30.671994 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:34:30.671994 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:34:30.671994 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:34:30.671994 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:34:30.673101 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:34:30.673101 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:34:30.673101 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:34:30.673101 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 600/1224] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:34:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:34:30.677804 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f12b8-010a-43b1-8c84-c603a6a09e0b None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:30.777600 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f12b8-010a-43b1-8c84-c603a6a09e0b None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:30.810215 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf4bb1-70b8-4476-867a-884263856b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:30.810813 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf4bb1-70b8-4476-867a-884263856b2d None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:30.811094 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf4bb1-70b8-4476-867a-884263856b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:30.811387 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf4bb1-70b8-4476-867a-884263856b2d None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:30.812016 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cf4bb1-70b8-4476-867a-884263856b2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:34:30.812317 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cf4bb1-70b8-4476-867a-884263856b2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:30.813652 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cf4bb1-70b8-4476-867a-884263856b2d None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:30.825035 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 625/1225] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:34:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:34:30.831551 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8174c7-ae39-4fc0-914c-3d975bc56744 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:30.925364 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:30.931901 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8174c7-ae39-4fc0-914c-3d975bc56744 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:30.965382 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db21f0ed-5901-4897-bae1-8ddc9240cb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:30.966005 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db21f0ed-5901-4897-bae1-8ddc9240cb20 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:30.966331 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db21f0ed-5901-4897-bae1-8ddc9240cb20 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:30.966643 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db21f0ed-5901-4897-bae1-8ddc9240cb20 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:30.973622 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db21f0ed-5901-4897-bae1-8ddc9240cb20 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:30.975407 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db21f0ed-5901-4897-bae1-8ddc9240cb20 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:30.987119 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-db21f0ed-5901-4897-bae1-8ddc9240cb20 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 03 12:34:30.987119 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:34:30.987119 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:34:30.987119 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:34:30.987119 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:34:30.987119 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:34:30.987119 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:34:30.987119 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:34:30.987119 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:34:30.987119 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:34:30.987119 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:34:30.987119 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:34:30.987119 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 12:34:30.987119 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 12:34:30.987119 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 12:34:30.987119 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 12:34:30.992128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:34:30.992128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:34:30.992128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 12:34:30.992128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:34:30.992128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 12:34:30.992128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:34:30.992128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:34:30.992128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:34:30.992128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:34:30.992128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:34:30.992128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:34:30.992128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:34:30.992128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:34:30.992128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:34:30.992128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:34:30.992128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:34:30.992128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:34:30.992128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:34:30.993307 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:34:30.993307 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:34:30.993307 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 12:34:30.993307 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 12:34:30.993307 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 12:34:30.993307 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 12:34:30.993307 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 12:34:30.993307 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 12:34:30.993307 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 03 12:34:30.993307 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:34:30.993307 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 601/1226] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:34:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:34:30.994513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf4bb1-70b8-4476-867a-884263856b2d None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:31.040595 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:31.092351 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf4bb1-70b8-4476-867a-884263856b2d None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:31.125904 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf126a-04b0-4149-8140-408c1f3909b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:31.126521 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf126a-04b0-4149-8140-408c1f3909b9 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:31.126806 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf126a-04b0-4149-8140-408c1f3909b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:31.127080 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf126a-04b0-4149-8140-408c1f3909b9 None admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:31.127856 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bf126a-04b0-4149-8140-408c1f3909b9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:34:31.128166 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bf126a-04b0-4149-8140-408c1f3909b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:31.129586 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bf126a-04b0-4149-8140-408c1f3909b9 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:31.147156 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 626/1227] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Oct 3 12:34:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:34:31.153043 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db21f0ed-5901-4897-bae1-8ddc9240cb20 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:31.255838 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db21f0ed-5901-4897-bae1-8ddc9240cb20 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:31.289763 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3189a3f-ebb2-4d02-8bfc-fb62e407b216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:31.290394 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3189a3f-ebb2-4d02-8bfc-fb62e407b216 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:31.290721 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3189a3f-ebb2-4d02-8bfc-fb62e407b216 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:31.290947 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3189a3f-ebb2-4d02-8bfc-fb62e407b216 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:31.301466 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3189a3f-ebb2-4d02-8bfc-fb62e407b216 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:31.307085 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3189a3f-ebb2-4d02-8bfc-fb62e407b216 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:31.316687 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-b3189a3f-ebb2-4d02-8bfc-fb62e407b216 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:34:31.316687 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:34:31.316687 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:34:31.316687 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:34:31.316687 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:34:31.316687 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:34:31.316687 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:34:31.316687 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:34:31.316687 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:34:31.316687 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:34:31.316687 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:34:31.316687 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:34:31.316687 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:34:31.316687 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:34:31.316687 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:34:31.316687 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:34:31.322482 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:34:31.322482 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:34:31.322482 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:34:31.322482 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:34:31.322482 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:34:31.322482 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:34:31.322482 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:34:31.322482 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:34:31.322482 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:34:31.322482 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:34:31.322482 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:34:31.322482 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:34:31.322482 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:34:31.322482 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:34:31.322482 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:34:31.322482 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:34:31.322482 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:34:31.335271 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:34:31.335271 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:34:31.335271 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:34:31.335271 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 602/1228] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:34:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:34:31.335577 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf126a-04b0-4149-8140-408c1f3909b9 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:31.421863 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf126a-04b0-4149-8140-408c1f3909b9 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:31.424161 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:31.455533 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d39c0e-b344-421b-9edc-f3a3afce18d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:31.456137 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d39c0e-b344-421b-9edc-f3a3afce18d1 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:31.456416 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d39c0e-b344-421b-9edc-f3a3afce18d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:31.456687 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d39c0e-b344-421b-9edc-f3a3afce18d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:31.457316 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d39c0e-b344-421b-9edc-f3a3afce18d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:34:31.457620 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d39c0e-b344-421b-9edc-f3a3afce18d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:31.458962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d39c0e-b344-421b-9edc-f3a3afce18d1 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:31.470384 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 627/1229] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:34:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:34:31.476329 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3189a3f-ebb2-4d02-8bfc-fb62e407b216 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:31.486651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:31.574895 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3189a3f-ebb2-4d02-8bfc-fb62e407b216 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:31.606318 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02651617-cdf9-4181-8ad9-e6ee630e7fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:31.607471 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02651617-cdf9-4181-8ad9-e6ee630e7fde None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:31.607771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02651617-cdf9-4181-8ad9-e6ee630e7fde None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:31.608087 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02651617-cdf9-4181-8ad9-e6ee630e7fde None admin] PATH_INFO: `/v3/projects/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:31.614431 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02651617-cdf9-4181-8ad9-e6ee630e7fde None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:31.615911 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02651617-cdf9-4181-8ad9-e6ee630e7fde None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:31.629230 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-02651617-cdf9-4181-8ad9-e6ee630e7fde None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:34:31.629230 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:34:31.629230 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:34:31.629230 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:34:31.629230 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:34:31.629230 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:34:31.629230 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:34:31.629230 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:34:31.629230 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:34:31.629230 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:34:31.629230 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:34:31.629230 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:34:31.629230 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 12:34:31.629230 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 12:34:31.629230 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 12:34:31.630188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 12:34:31.630188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:34:31.630188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:34:31.630188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:34:31.630188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:34:31.630188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:34:31.630188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:34:31.630188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:34:31.630188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:34:31.630188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:34:31.630188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:34:31.630188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:34:31.630188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:34:31.630188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:34:31.630188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:34:31.630188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:34:31.630188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:34:31.631113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 12:34:31.631113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 12:34:31.631113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 12:34:31.631113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 12:34:31.631113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 12:34:31.631113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 12:34:31.631113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:34:31.631113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:34:31.631113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 603/1230] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:34:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:34:31.636446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d39c0e-b344-421b-9edc-f3a3afce18d1 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:31.733811 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d39c0e-b344-421b-9edc-f3a3afce18d1 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:31.765835 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f45cb-bf0c-45e6-9ab9-2c531c414487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:31.766445 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f45cb-bf0c-45e6-9ab9-2c531c414487 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:31.766724 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f45cb-bf0c-45e6-9ab9-2c531c414487 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:31.766994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f45cb-bf0c-45e6-9ab9-2c531c414487 None admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:31.767703 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4f45cb-bf0c-45e6-9ab9-2c531c414487 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:34:31.768008 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4f45cb-bf0c-45e6-9ab9-2c531c414487 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:31.769451 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4f45cb-bf0c-45e6-9ab9-2c531c414487 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:31.781789 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 628/1231] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 12:34:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:34:31.787741 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02651617-cdf9-4181-8ad9-e6ee630e7fde None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:31.852050 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:31.885754 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02651617-cdf9-4181-8ad9-e6ee630e7fde None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:31.918407 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f0e2b5-8cc0-4f82-b505-7d29cac20858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:31.919053 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0e2b5-8cc0-4f82-b505-7d29cac20858 None admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:31.919462 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0e2b5-8cc0-4f82-b505-7d29cac20858 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:31.919919 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0e2b5-8cc0-4f82-b505-7d29cac20858 None admin] PATH_INFO: `/v3/projects/4ea2fa8ad01240f7859182ebbe3051c7/users/25eef64e92e9422ea62508c3eb0a197e/roles/ff6b2d4413644db39a2bd8576afc6ccb` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:31.937049 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f0e2b5-8cc0-4f82-b505-7d29cac20858 None admin] RBAC: Authorizing `identity:create_grant(project_id=4ea2fa8ad01240f7859182ebbe3051c7, user_id=25eef64e92e9422ea62508c3eb0a197e, role_id=ff6b2d4413644db39a2bd8576afc6ccb)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:31.939106 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f0e2b5-8cc0-4f82-b505-7d29cac20858 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:32.039285 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 604/1232] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 12:34:31 2022] PUT /identity/v3/projects/4ea2fa8ad01240f7859182ebbe3051c7/users/25eef64e92e9422ea62508c3eb0a197e/roles/ff6b2d4413644db39a2bd8576afc6ccb => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:34:34.172667 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0dd4d3-1dd0-461f-ac8a-b55a823ca808 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:34.173236 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0dd4d3-1dd0-461f-ac8a-b55a823ca808 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:34.173236 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0dd4d3-1dd0-461f-ac8a-b55a823ca808 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:34.174529 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 629/1233] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:34:34.179610 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f7ba9-8b86-4cb0-80a8-408d48f71331 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:34.180004 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f7ba9-8b86-4cb0-80a8-408d48f71331 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:34.180152 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f7ba9-8b86-4cb0-80a8-408d48f71331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:34.198583 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-443f7ba9-8b86-4cb0-80a8-408d48f71331 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:34:34.199802 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-443f7ba9-8b86-4cb0-80a8-408d48f71331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:34:34.266882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 605/1234] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:34:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:34:34.535342 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0dd4d3-1dd0-461f-ac8a-b55a823ca808 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:34.559606 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:34.635451 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0dd4d3-1dd0-461f-ac8a-b55a823ca808 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:34.670022 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04c6896-8a45-4d04-a9ec-ef34a547d849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:34.670858 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c6896-8a45-4d04-a9ec-ef34a547d849 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:34.671136 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c6896-8a45-4d04-a9ec-ef34a547d849 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:34.671447 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c6896-8a45-4d04-a9ec-ef34a547d849 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:34.681471 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04c6896-8a45-4d04-a9ec-ef34a547d849 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:34:34.686198 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-c04c6896-8a45-4d04-a9ec-ef34a547d849 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:34:34.686198 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:34:34.686198 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:34:34.686198 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:34:34.686198 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:34:34.686198 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:34:34.686198 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:34:34.686198 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:34:34.686198 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:34:34.686198 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:34:34.686198 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:34:34.686198 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:34:34.686198 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:34:34.686198 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:34:34.686198 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:34:34.686198 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:34:34.688111 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:34:34.688111 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:34:34.688111 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:34:34.688111 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:34:34.688111 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:34:34.688111 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:34:34.688111 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:34:34.688111 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:34:34.688111 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:34:34.688111 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:34:34.688111 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:34:34.688111 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:34:34.688111 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:34:34.688111 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:34:34.688111 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:34:34.688111 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:34:34.689139 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:34:34.689139 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:34:34.689139 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:34:34.689139 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:34:34.689139 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:34:34.689139 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:34:34.689139 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:34:34.689139 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:34:34.689139 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:34:34.689139 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:34:34.689139 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 12:34:34.689139 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:34:34.689139 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 630/1235] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 12:34:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:34:34.693301 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f7ba9-8b86-4cb0-80a8-408d48f71331 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:34.740012 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:34.791882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f7ba9-8b86-4cb0-80a8-408d48f71331 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:34.824656 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa34a7-44a4-4201-8f35-4814aaac3301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:34.825265 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa34a7-44a4-4201-8f35-4814aaac3301 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:34.825557 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa34a7-44a4-4201-8f35-4814aaac3301 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:34.826416 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa34a7-44a4-4201-8f35-4814aaac3301 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:34.827078 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa34a7-44a4-4201-8f35-4814aaac3301 None admin] RBAC: Adding query filter params (name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:34:34.827411 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa34a7-44a4-4201-8f35-4814aaac3301 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:34.828787 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa34a7-44a4-4201-8f35-4814aaac3301 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:34.836448 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 606/1236] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 12:34:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:34:34.842964 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04c6896-8a45-4d04-a9ec-ef34a547d849 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:34.929364 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:34.945162 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04c6896-8a45-4d04-a9ec-ef34a547d849 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:34.977613 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12593531-3ad6-4b10-96b7-1a5f96ef2f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:34.978234 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12593531-3ad6-4b10-96b7-1a5f96ef2f16 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:34.978510 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12593531-3ad6-4b10-96b7-1a5f96ef2f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:34.978776 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12593531-3ad6-4b10-96b7-1a5f96ef2f16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:34.988762 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12593531-3ad6-4b10-96b7-1a5f96ef2f16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:34.990180 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12593531-3ad6-4b10-96b7-1a5f96ef2f16 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:35.004095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-12593531-3ad6-4b10-96b7-1a5f96ef2f16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:34:35.004095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:34:35.004095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:34:35.004095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:34:35.004095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:34:35.004095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:34:35.004095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:34:35.004095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:34:35.004095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:34:35.004095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:34:35.004095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:34:35.004095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:34:35.004095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:34:35.004095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:34:35.004095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:34:35.004095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:34:35.005249 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:34:35.005249 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:34:35.005249 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:34:35.005249 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:34:35.005249 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:34:35.005249 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:34:35.005249 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:34:35.005249 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:34:35.005249 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:34:35.005249 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:34:35.005249 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:34:35.005249 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:34:35.005249 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:34:35.005249 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:34:35.005249 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:34:35.005249 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:34:35.005249 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:34:35.006147 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:34:35.006147 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:34:35.006147 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:34:35.006147 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 631/1237] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:34:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:34:35.011019 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa34a7-44a4-4201-8f35-4814aaac3301 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:35.110524 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa34a7-44a4-4201-8f35-4814aaac3301 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:35.120674 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:35.145385 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6675ae4a-094d-42fe-a6b7-dd55e56cdfa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:35.147251 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6675ae4a-094d-42fe-a6b7-dd55e56cdfa7 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:35.147251 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6675ae4a-094d-42fe-a6b7-dd55e56cdfa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:35.147251 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6675ae4a-094d-42fe-a6b7-dd55e56cdfa7 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:35.147251 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6675ae4a-094d-42fe-a6b7-dd55e56cdfa7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:34:35.147587 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6675ae4a-094d-42fe-a6b7-dd55e56cdfa7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:35.148819 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6675ae4a-094d-42fe-a6b7-dd55e56cdfa7 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:35.160208 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 607/1238] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:34:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:34:35.165991 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12593531-3ad6-4b10-96b7-1a5f96ef2f16 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:35.266025 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12593531-3ad6-4b10-96b7-1a5f96ef2f16 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:35.298770 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d0a6e5-d701-4ce4-a769-e6e0256acef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:35.299411 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d0a6e5-d701-4ce4-a769-e6e0256acef2 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:35.299693 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d0a6e5-d701-4ce4-a769-e6e0256acef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:35.299961 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d0a6e5-d701-4ce4-a769-e6e0256acef2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:35.302560 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:35.307497 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d0a6e5-d701-4ce4-a769-e6e0256acef2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:35.308901 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d0a6e5-d701-4ce4-a769-e6e0256acef2 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:35.319788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-83d0a6e5-d701-4ce4-a769-e6e0256acef2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 03 12:34:35.319788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:34:35.319788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:34:35.319788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:34:35.319788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:34:35.319788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:34:35.319788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:34:35.319788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:34:35.319788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:34:35.319788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:34:35.319788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:34:35.319788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:34:35.319788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 12:34:35.319788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 12:34:35.319788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 12:34:35.319788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 12:34:35.320936 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:34:35.320936 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:34:35.320936 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 12:34:35.320936 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:34:35.320936 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 12:34:35.320936 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:34:35.320936 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:34:35.320936 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:34:35.320936 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:34:35.320936 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:34:35.320936 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:34:35.320936 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:34:35.320936 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:34:35.320936 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:34:35.320936 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:34:35.320936 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:34:35.320936 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:34:35.320936 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:34:35.321934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:34:35.321934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:34:35.321934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 12:34:35.321934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 12:34:35.321934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 12:34:35.321934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 12:34:35.321934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 12:34:35.321934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 12:34:35.321934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 03 12:34:35.321934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:34:35.321934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 632/1239] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:34:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:34:35.327229 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6675ae4a-094d-42fe-a6b7-dd55e56cdfa7 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:35.440857 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6675ae4a-094d-42fe-a6b7-dd55e56cdfa7 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:35.475136 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc4df2-96a1-4f85-bbd1-9d4d09dd1339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:35.476013 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc4df2-96a1-4f85-bbd1-9d4d09dd1339 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:35.476285 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc4df2-96a1-4f85-bbd1-9d4d09dd1339 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:35.476554 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc4df2-96a1-4f85-bbd1-9d4d09dd1339 None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:35.477250 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc4df2-96a1-4f85-bbd1-9d4d09dd1339 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:34:35.477545 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc4df2-96a1-4f85-bbd1-9d4d09dd1339 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:35.478943 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc4df2-96a1-4f85-bbd1-9d4d09dd1339 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:35.495704 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 608/1240] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Oct 3 12:34:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:34:35.502418 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d0a6e5-d701-4ce4-a769-e6e0256acef2 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:35.601692 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d0a6e5-d701-4ce4-a769-e6e0256acef2 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:35.634685 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b218cc-43ac-4eb0-bfb3-474517e9633b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:35.635289 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b218cc-43ac-4eb0-bfb3-474517e9633b None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:35.635603 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b218cc-43ac-4eb0-bfb3-474517e9633b None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:35.635874 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b218cc-43ac-4eb0-bfb3-474517e9633b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:35.694907 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:35.704267 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b218cc-43ac-4eb0-bfb3-474517e9633b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:35.705705 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b218cc-43ac-4eb0-bfb3-474517e9633b None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:35.805576 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 633/1241] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:34:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 12:34:35.812365 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc4df2-96a1-4f85-bbd1-9d4d09dd1339 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:35.840806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:35.910225 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc4df2-96a1-4f85-bbd1-9d4d09dd1339 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:35.945786 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394dc474-37f2-4c86-b11c-2de154fa41b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:35.946532 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394dc474-37f2-4c86-b11c-2de154fa41b0 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:35.946902 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394dc474-37f2-4c86-b11c-2de154fa41b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:35.947259 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394dc474-37f2-4c86-b11c-2de154fa41b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:35.957969 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394dc474-37f2-4c86-b11c-2de154fa41b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:35.959434 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394dc474-37f2-4c86-b11c-2de154fa41b0 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:35.974489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-394dc474-37f2-4c86-b11c-2de154fa41b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:34:35.974489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:34:35.974489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:34:35.974489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:34:35.974489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:34:35.974489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:34:35.974489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:34:35.974489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:34:35.974489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:34:35.974489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:34:35.974489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:34:35.974489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:34:35.974489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:34:35.974489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:34:35.974489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:34:35.974489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:34:35.975981 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:34:35.975981 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:34:35.975981 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:34:35.975981 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:34:35.975981 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:34:35.975981 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:34:35.975981 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:34:35.975981 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:34:35.975981 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:34:35.975981 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:34:35.975981 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:34:35.975981 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:34:35.975981 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:34:35.975981 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:34:35.975981 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:34:35.975981 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:34:35.975981 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:34:35.977054 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:34:35.977054 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:34:35.977054 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:34:35.977054 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 609/1242] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:34:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:34:35.981709 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b218cc-43ac-4eb0-bfb3-474517e9633b None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:36.076311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:36.081768 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b218cc-43ac-4eb0-bfb3-474517e9633b None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:36.114315 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624e95b-9ac1-4181-bd0d-3c995f22bb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:36.114912 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624e95b-9ac1-4181-bd0d-3c995f22bb67 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:36.115191 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624e95b-9ac1-4181-bd0d-3c995f22bb67 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:36.115489 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624e95b-9ac1-4181-bd0d-3c995f22bb67 None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:36.116213 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2624e95b-9ac1-4181-bd0d-3c995f22bb67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:34:36.116514 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2624e95b-9ac1-4181-bd0d-3c995f22bb67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:36.117906 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2624e95b-9ac1-4181-bd0d-3c995f22bb67 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:36.133361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 634/1243] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:34:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:34:36.135196 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394dc474-37f2-4c86-b11c-2de154fa41b0 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:36.211829 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:36.234059 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394dc474-37f2-4c86-b11c-2de154fa41b0 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:36.268555 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21902fda-c61f-4579-b373-ba6b8aea0717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:36.269179 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21902fda-c61f-4579-b373-ba6b8aea0717 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:36.269458 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21902fda-c61f-4579-b373-ba6b8aea0717 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:36.269725 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21902fda-c61f-4579-b373-ba6b8aea0717 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:36.276109 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21902fda-c61f-4579-b373-ba6b8aea0717 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:36.277534 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21902fda-c61f-4579-b373-ba6b8aea0717 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:36.288707 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-21902fda-c61f-4579-b373-ba6b8aea0717 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:34:36.288707 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:34:36.288707 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:34:36.288707 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:34:36.288707 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:34:36.288707 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:34:36.288707 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:34:36.288707 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:34:36.288707 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:34:36.288707 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:34:36.288707 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:34:36.288707 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:34:36.288707 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 12:34:36.288707 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 12:34:36.288707 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 12:34:36.289818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 12:34:36.289818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:34:36.289818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:34:36.289818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:34:36.289818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:34:36.289818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:34:36.289818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:34:36.289818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:34:36.289818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:34:36.289818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:34:36.289818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:34:36.289818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:34:36.289818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:34:36.289818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:34:36.289818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:34:36.289818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:34:36.289818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:34:36.290922 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 12:34:36.290922 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 12:34:36.290922 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 12:34:36.290922 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 12:34:36.290922 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 12:34:36.290922 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 12:34:36.290922 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:34:36.290922 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:34:36.290922 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 610/1244] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:34:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:34:36.297704 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624e95b-9ac1-4181-bd0d-3c995f22bb67 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:36.402669 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624e95b-9ac1-4181-bd0d-3c995f22bb67 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:36.437105 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5be559-a505-4af6-9f44-64aee28420fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:36.437711 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5be559-a505-4af6-9f44-64aee28420fc None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:36.437998 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5be559-a505-4af6-9f44-64aee28420fc None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:36.438252 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5be559-a505-4af6-9f44-64aee28420fc None admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:36.438926 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5be559-a505-4af6-9f44-64aee28420fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:34:36.439228 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5be559-a505-4af6-9f44-64aee28420fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:36.440659 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5be559-a505-4af6-9f44-64aee28420fc None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:36.452288 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 635/1245] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 12:34:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:34:36.458437 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21902fda-c61f-4579-b373-ba6b8aea0717 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:36.559236 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21902fda-c61f-4579-b373-ba6b8aea0717 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:36.592884 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa10f9d-fde1-458b-b7e8-41c1d1da2d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:36.593482 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa10f9d-fde1-458b-b7e8-41c1d1da2d97 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:36.593760 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa10f9d-fde1-458b-b7e8-41c1d1da2d97 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:36.594031 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa10f9d-fde1-458b-b7e8-41c1d1da2d97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:36.594744 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa10f9d-fde1-458b-b7e8-41c1d1da2d97 None admin] RBAC: Adding query filter params (role.id=ff6b2d4413644db39a2bd8576afc6ccb, scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7, user.id=25eef64e92e9422ea62508c3eb0a197e) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:34:36.595045 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa10f9d-fde1-458b-b7e8-41c1d1da2d97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:36.596492 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa10f9d-fde1-458b-b7e8-41c1d1da2d97 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:36.602368 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 611/1246] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:34:36 2022] GET /identity/v3/role_assignments?user.id=25eef64e92e9422ea62508c3eb0a197e&scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7&role.id=ff6b2d4413644db39a2bd8576afc6ccb => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:34:38.824908 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4413e80-fa2f-4405-bb5f-98ab113a4346 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:38.825498 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4413e80-fa2f-4405-bb5f-98ab113a4346 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:38.825498 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4413e80-fa2f-4405-bb5f-98ab113a4346 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:38.826827 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 636/1247] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:34:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:34:38.832089 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abf73e-423c-46f6-a7d8-7c5da58b147c None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:38.832471 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abf73e-423c-46f6-a7d8-7c5da58b147c None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:38.832668 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abf73e-423c-46f6-a7d8-7c5da58b147c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:38.838924 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:38.851633 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-a9abf73e-423c-46f6-a7d8-7c5da58b147c None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:34:38.852816 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a9abf73e-423c-46f6-a7d8-7c5da58b147c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:34:38.917704 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 612/1248] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:34:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:34:39.191916 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4413e80-fa2f-4405-bb5f-98ab113a4346 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:39.209794 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:39.291184 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4413e80-fa2f-4405-bb5f-98ab113a4346 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:39.324802 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373e33e-8ea3-424e-b76d-452232ba3d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:39.325452 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373e33e-8ea3-424e-b76d-452232ba3d23 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:39.325783 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373e33e-8ea3-424e-b76d-452232ba3d23 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:39.326054 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373e33e-8ea3-424e-b76d-452232ba3d23 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:39.335771 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b373e33e-8ea3-424e-b76d-452232ba3d23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:34:39.340479 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-b373e33e-8ea3-424e-b76d-452232ba3d23 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:34:39.340479 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:34:39.340479 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:34:39.340479 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:34:39.340479 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:34:39.340479 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:34:39.340479 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:34:39.340479 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:34:39.340479 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:34:39.340479 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:34:39.340479 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:34:39.340479 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:34:39.340479 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:34:39.340479 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:34:39.340479 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:34:39.340479 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:34:39.344859 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:34:39.344859 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:34:39.344859 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:34:39.344859 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:34:39.344859 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:34:39.344859 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:34:39.344859 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:34:39.344859 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:34:39.344859 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:34:39.344859 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:34:39.344859 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:34:39.344859 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:34:39.344859 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:34:39.344859 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:34:39.344859 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:34:39.344859 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:34:39.345711 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:34:39.345711 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:34:39.345711 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:34:39.345711 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:34:39.345711 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:34:39.345711 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:34:39.345711 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:34:39.345711 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:34:39.345711 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:34:39.345711 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:34:39.345711 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:34:39.345711 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:34:39.345711 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 637/1249] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 12:34:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:34:39.348431 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9abf73e-423c-46f6-a7d8-7c5da58b147c None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:39.446179 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9abf73e-423c-46f6-a7d8-7c5da58b147c None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:39.474862 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:39.479893 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c70cf-afa9-4a81-9a6b-4b0a7638192f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:39.480496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c70cf-afa9-4a81-9a6b-4b0a7638192f None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:39.480773 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c70cf-afa9-4a81-9a6b-4b0a7638192f None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:39.481043 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c70cf-afa9-4a81-9a6b-4b0a7638192f None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:39.481692 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25c70cf-afa9-4a81-9a6b-4b0a7638192f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:34:39.482580 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25c70cf-afa9-4a81-9a6b-4b0a7638192f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:39.484495 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25c70cf-afa9-4a81-9a6b-4b0a7638192f None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:39.492322 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 613/1250] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:34:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:34:39.498912 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373e33e-8ea3-424e-b76d-452232ba3d23 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:39.574023 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:39.597239 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373e33e-8ea3-424e-b76d-452232ba3d23 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:39.629995 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06e21a-41af-40cb-8306-a9b780e9f286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:39.630622 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06e21a-41af-40cb-8306-a9b780e9f286 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:39.630904 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06e21a-41af-40cb-8306-a9b780e9f286 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:39.631177 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06e21a-41af-40cb-8306-a9b780e9f286 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:39.642136 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a06e21a-41af-40cb-8306-a9b780e9f286 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:39.643593 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a06e21a-41af-40cb-8306-a9b780e9f286 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:39.658513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-7a06e21a-41af-40cb-8306-a9b780e9f286 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:34:39.658513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:34:39.658513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:34:39.658513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:34:39.658513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:34:39.658513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:34:39.658513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:34:39.658513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:34:39.658513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:34:39.658513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:34:39.658513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:34:39.658513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:34:39.658513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:34:39.658513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:34:39.658513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:34:39.658513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:34:39.659682 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:34:39.659682 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:34:39.659682 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:34:39.659682 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:34:39.659682 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:34:39.659682 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:34:39.659682 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:34:39.659682 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:34:39.659682 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:34:39.659682 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:34:39.659682 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:34:39.659682 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:34:39.659682 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:34:39.659682 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:34:39.659682 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:34:39.659682 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:34:39.659682 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:34:39.660881 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:34:39.660881 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:34:39.660881 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:34:39.660881 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 638/1251] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:34:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:34:39.665605 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c70cf-afa9-4a81-9a6b-4b0a7638192f None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:39.766290 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c70cf-afa9-4a81-9a6b-4b0a7638192f None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:39.799589 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e1fa4-f65a-48f3-aef5-d9ebacaca63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:39.800190 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e1fa4-f65a-48f3-aef5-d9ebacaca63d None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:39.800466 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e1fa4-f65a-48f3-aef5-d9ebacaca63d None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:39.800737 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e1fa4-f65a-48f3-aef5-d9ebacaca63d None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:39.801363 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083e1fa4-f65a-48f3-aef5-d9ebacaca63d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:34:39.801662 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083e1fa4-f65a-48f3-aef5-d9ebacaca63d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:39.802997 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083e1fa4-f65a-48f3-aef5-d9ebacaca63d None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:39.814431 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 614/1252] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:34:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:34:39.820404 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06e21a-41af-40cb-8306-a9b780e9f286 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:39.925205 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06e21a-41af-40cb-8306-a9b780e9f286 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:39.950465 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:39.958054 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011820c-19b9-48ed-8bf1-8c3f9099f5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:39.958683 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011820c-19b9-48ed-8bf1-8c3f9099f5ea None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:39.958964 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011820c-19b9-48ed-8bf1-8c3f9099f5ea None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:39.959236 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011820c-19b9-48ed-8bf1-8c3f9099f5ea None admin] PATH_INFO: `/v3/users/placement` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:39.966952 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a011820c-19b9-48ed-8bf1-8c3f9099f5ea None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:39.968631 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a011820c-19b9-48ed-8bf1-8c3f9099f5ea None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:39.979773 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-a011820c-19b9-48ed-8bf1-8c3f9099f5ea None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 03 12:34:39.979773 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:34:39.979773 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:34:39.979773 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:34:39.979773 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:34:39.979773 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:34:39.979773 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:34:39.979773 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:34:39.979773 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:34:39.979773 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:34:39.979773 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:34:39.979773 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:34:39.979773 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 12:34:39.979773 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 12:34:39.979773 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 12:34:39.979773 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 12:34:39.984275 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:34:39.984275 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:34:39.984275 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 12:34:39.984275 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:34:39.984275 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 12:34:39.984275 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:34:39.984275 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:34:39.984275 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:34:39.984275 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:34:39.984275 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:34:39.984275 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:34:39.984275 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:34:39.984275 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:34:39.984275 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:34:39.984275 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:34:39.984275 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:34:39.984275 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:34:39.984275 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:34:39.986057 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:34:39.986057 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:34:39.986057 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 12:34:39.986057 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 12:34:39.986057 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 12:34:39.986057 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 12:34:39.986057 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 12:34:39.986057 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 12:34:39.986057 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 03 12:34:39.986057 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:34:39.986057 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 639/1253] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:34:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:34:39.990261 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e1fa4-f65a-48f3-aef5-d9ebacaca63d None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:40.046667 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:40.092890 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e1fa4-f65a-48f3-aef5-d9ebacaca63d None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:40.126466 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d84a587-fc6e-4c18-bfac-643d899c81a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:40.127072 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84a587-fc6e-4c18-bfac-643d899c81a4 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:40.127376 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84a587-fc6e-4c18-bfac-643d899c81a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:40.127653 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84a587-fc6e-4c18-bfac-643d899c81a4 None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:40.128370 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d84a587-fc6e-4c18-bfac-643d899c81a4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:34:40.128674 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d84a587-fc6e-4c18-bfac-643d899c81a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:40.130160 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d84a587-fc6e-4c18-bfac-643d899c81a4 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:40.147579 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 615/1254] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Oct 3 12:34:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:34:40.153440 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011820c-19b9-48ed-8bf1-8c3f9099f5ea None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:40.251379 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011820c-19b9-48ed-8bf1-8c3f9099f5ea None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:40.284654 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a080878b-52dd-4a15-a0e0-ad79a042bdd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:40.285262 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080878b-52dd-4a15-a0e0-ad79a042bdd5 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:40.285547 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080878b-52dd-4a15-a0e0-ad79a042bdd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:40.285822 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080878b-52dd-4a15-a0e0-ad79a042bdd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:40.339958 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:40.352389 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a080878b-52dd-4a15-a0e0-ad79a042bdd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:40.353867 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a080878b-52dd-4a15-a0e0-ad79a042bdd5 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:40.450262 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 640/1255] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:34:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 12:34:40.456463 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d84a587-fc6e-4c18-bfac-643d899c81a4 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:40.555887 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d84a587-fc6e-4c18-bfac-643d899c81a4 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:40.569598 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:40.591129 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ab518-ee9f-46a2-9141-60265e8f3c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:40.592747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ab518-ee9f-46a2-9141-60265e8f3c71 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:40.593026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ab518-ee9f-46a2-9141-60265e8f3c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:40.593295 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ab518-ee9f-46a2-9141-60265e8f3c71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:40.603558 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ab518-ee9f-46a2-9141-60265e8f3c71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:40.604989 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ab518-ee9f-46a2-9141-60265e8f3c71 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:40.619649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-ec6ab518-ee9f-46a2-9141-60265e8f3c71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:34:40.619649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:34:40.619649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:34:40.619649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:34:40.619649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:34:40.619649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:34:40.619649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:34:40.619649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:34:40.619649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:34:40.619649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:34:40.619649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:34:40.619649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:34:40.619649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:34:40.619649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:34:40.619649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:34:40.619649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:34:40.620775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:34:40.620775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:34:40.620775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:34:40.620775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:34:40.620775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:34:40.620775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:34:40.620775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:34:40.620775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:34:40.620775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:34:40.620775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:34:40.620775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:34:40.620775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:34:40.620775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:34:40.620775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:34:40.620775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:34:40.620775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:34:40.620775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:34:40.621900 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:34:40.621900 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:34:40.621900 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:34:40.621900 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 616/1256] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:34:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:34:40.627254 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a080878b-52dd-4a15-a0e0-ad79a042bdd5 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:40.714092 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:40.725898 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a080878b-52dd-4a15-a0e0-ad79a042bdd5 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:40.758592 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a2c06a-1443-407d-b588-b387e2a5c4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:40.759194 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2c06a-1443-407d-b588-b387e2a5c4d3 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:40.759512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2c06a-1443-407d-b588-b387e2a5c4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:40.759786 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2c06a-1443-407d-b588-b387e2a5c4d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:40.760417 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a2c06a-1443-407d-b588-b387e2a5c4d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:34:40.760722 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a2c06a-1443-407d-b588-b387e2a5c4d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:40.762078 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a2c06a-1443-407d-b588-b387e2a5c4d3 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:40.775018 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 641/1257] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:34:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:34:40.781244 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ab518-ee9f-46a2-9141-60265e8f3c71 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:40.882079 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ab518-ee9f-46a2-9141-60265e8f3c71 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:40.915708 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f5524-d122-4c11-af4f-01164c20ec5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:40.916324 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f5524-d122-4c11-af4f-01164c20ec5e None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:40.916595 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f5524-d122-4c11-af4f-01164c20ec5e None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:40.916860 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f5524-d122-4c11-af4f-01164c20ec5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:40.923761 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0f5524-d122-4c11-af4f-01164c20ec5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:40.925215 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0f5524-d122-4c11-af4f-01164c20ec5e None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:40.936570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-8f0f5524-d122-4c11-af4f-01164c20ec5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:34:40.936570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:34:40.936570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:34:40.936570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:34:40.936570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:34:40.936570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:34:40.936570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:34:40.936570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:34:40.936570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:34:40.936570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:34:40.936570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:34:40.936570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:34:40.936570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 12:34:40.936570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 12:34:40.936570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 12:34:40.937628 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 12:34:40.937628 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:34:40.937628 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:34:40.937628 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:34:40.937628 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:34:40.937628 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:34:40.937628 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:34:40.937628 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:34:40.937628 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:34:40.937628 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:34:40.937628 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:34:40.937628 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:34:40.937628 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:34:40.937628 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:34:40.937628 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:34:40.937628 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:34:40.937628 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:34:40.938756 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 12:34:40.938756 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 12:34:40.938756 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 12:34:40.938756 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 12:34:40.938756 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 12:34:40.938756 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 12:34:40.938756 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:34:40.938756 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:34:40.938756 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 617/1258] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:34:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:34:40.943702 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a2c06a-1443-407d-b588-b387e2a5c4d3 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:41.041283 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a2c06a-1443-407d-b588-b387e2a5c4d3 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:41.074437 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53744f24-4bf8-49ec-a77a-b61c6a1f2629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:41.075035 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53744f24-4bf8-49ec-a77a-b61c6a1f2629 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:41.075334 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53744f24-4bf8-49ec-a77a-b61c6a1f2629 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:41.076280 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53744f24-4bf8-49ec-a77a-b61c6a1f2629 None admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:41.077392 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53744f24-4bf8-49ec-a77a-b61c6a1f2629 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:34:41.077697 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53744f24-4bf8-49ec-a77a-b61c6a1f2629 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:41.079098 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53744f24-4bf8-49ec-a77a-b61c6a1f2629 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:41.091149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 642/1259] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 12:34:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:34:41.097856 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f5524-d122-4c11-af4f-01164c20ec5e None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:41.123154 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:41.196670 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f5524-d122-4c11-af4f-01164c20ec5e None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:41.231784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713def8a-0b91-48f2-a2d5-5375a6ffb785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:41.232736 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713def8a-0b91-48f2-a2d5-5375a6ffb785 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:41.233005 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713def8a-0b91-48f2-a2d5-5375a6ffb785 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:41.233272 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713def8a-0b91-48f2-a2d5-5375a6ffb785 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:41.233987 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713def8a-0b91-48f2-a2d5-5375a6ffb785 None admin] RBAC: Adding query filter params (role.id=7933c45dcdd54ef0a98018ba8a8bddaa, scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7, user.id=25eef64e92e9422ea62508c3eb0a197e) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:34:41.234289 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713def8a-0b91-48f2-a2d5-5375a6ffb785 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:41.235792 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713def8a-0b91-48f2-a2d5-5375a6ffb785 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:41.241239 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 618/1260] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:34:41 2022] GET /identity/v3/role_assignments?user.id=25eef64e92e9422ea62508c3eb0a197e&scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7&role.id=7933c45dcdd54ef0a98018ba8a8bddaa => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:34:43.374604 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3379c5-2b01-4bad-bc09-e41af4ec3fdb None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:43.375177 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3379c5-2b01-4bad-bc09-e41af4ec3fdb None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:43.375177 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3379c5-2b01-4bad-bc09-e41af4ec3fdb None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:43.376491 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 643/1261] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:34:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:34:43.381716 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449ab79-522d-47d5-a80d-46998205c1cb None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:43.382152 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449ab79-522d-47d5-a80d-46998205c1cb None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:43.382260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449ab79-522d-47d5-a80d-46998205c1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:43.400214 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-8449ab79-522d-47d5-a80d-46998205c1cb None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:34:43.401387 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-8449ab79-522d-47d5-a80d-46998205c1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:34:43.467077 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 619/1262] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:34:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 12:34:43.739247 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3379c5-2b01-4bad-bc09-e41af4ec3fdb None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:43.750112 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:43.839976 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3379c5-2b01-4bad-bc09-e41af4ec3fdb None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:43.876236 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c234ddff-d5f6-4876-a7e8-0cf5a12a3eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:43.876869 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234ddff-d5f6-4876-a7e8-0cf5a12a3eb1 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:43.877151 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234ddff-d5f6-4876-a7e8-0cf5a12a3eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:43.877423 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234ddff-d5f6-4876-a7e8-0cf5a12a3eb1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:43.887275 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c234ddff-d5f6-4876-a7e8-0cf5a12a3eb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:34:43.892764 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-c234ddff-d5f6-4876-a7e8-0cf5a12a3eb1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:34:43.892764 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:34:43.892764 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:34:43.892764 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:34:43.892764 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:34:43.892764 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:34:43.892764 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:34:43.892764 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:34:43.892764 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:34:43.892764 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:34:43.892764 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:34:43.892764 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:34:43.892764 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:34:43.892764 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:34:43.892764 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:34:43.892764 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:34:43.896851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:34:43.896851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:34:43.896851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:34:43.896851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:34:43.896851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:34:43.896851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:34:43.896851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:34:43.896851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:34:43.896851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:34:43.896851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:34:43.896851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:34:43.896851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:34:43.896851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:34:43.896851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:34:43.896851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:34:43.896851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:34:43.897910 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:34:43.897910 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:34:43.897910 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:34:43.897910 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:34:43.897910 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:34:43.897910 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:34:43.897910 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:34:43.897910 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:34:43.897910 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:34:43.897910 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:34:43.897910 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:34:43.897910 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:34:43.897910 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 644/1263] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 12:34:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:34:43.899905 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449ab79-522d-47d5-a80d-46998205c1cb None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:43.917972 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:43.995099 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449ab79-522d-47d5-a80d-46998205c1cb None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:44.034730 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f86757-2160-4894-92fd-ed4ad8fe4193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:44.035365 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f86757-2160-4894-92fd-ed4ad8fe4193 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:44.035648 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f86757-2160-4894-92fd-ed4ad8fe4193 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:44.035918 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f86757-2160-4894-92fd-ed4ad8fe4193 None admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:44.036567 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f86757-2160-4894-92fd-ed4ad8fe4193 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:34:44.036863 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f86757-2160-4894-92fd-ed4ad8fe4193 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:44.038218 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f86757-2160-4894-92fd-ed4ad8fe4193 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:44.046807 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 620/1264] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:34:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:34:44.052614 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c234ddff-d5f6-4876-a7e8-0cf5a12a3eb1 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:44.118680 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:44.152413 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c234ddff-d5f6-4876-a7e8-0cf5a12a3eb1 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:44.184786 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4592a48-d278-45ff-ba26-514669c0f97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:44.185434 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4592a48-d278-45ff-ba26-514669c0f97c None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:44.185712 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4592a48-d278-45ff-ba26-514669c0f97c None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:44.185983 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4592a48-d278-45ff-ba26-514669c0f97c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:44.196073 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4592a48-d278-45ff-ba26-514669c0f97c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:44.197483 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4592a48-d278-45ff-ba26-514669c0f97c None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:44.212129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-e4592a48-d278-45ff-ba26-514669c0f97c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:34:44.212129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:34:44.212129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:34:44.212129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:34:44.212129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:34:44.212129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:34:44.212129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:34:44.212129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:34:44.212129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:34:44.212129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:34:44.212129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:34:44.212129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:34:44.212129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:34:44.212129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:34:44.212129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:34:44.212129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:34:44.213333 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:34:44.213333 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:34:44.213333 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:34:44.213333 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:34:44.213333 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:34:44.213333 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:34:44.213333 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:34:44.213333 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:34:44.213333 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:34:44.213333 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:34:44.213333 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:34:44.213333 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:34:44.213333 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:34:44.213333 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:34:44.213333 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:34:44.213333 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:34:44.213333 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:34:44.214612 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:34:44.214612 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:34:44.214612 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:34:44.214612 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 645/1265] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:34:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:34:44.219777 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f86757-2160-4894-92fd-ed4ad8fe4193 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:44.303632 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:44.322021 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f86757-2160-4894-92fd-ed4ad8fe4193 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:44.357040 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3f07d-702b-494e-84b9-42290eff878d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:44.357628 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3f07d-702b-494e-84b9-42290eff878d None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:44.357903 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3f07d-702b-494e-84b9-42290eff878d None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:44.358170 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3f07d-702b-494e-84b9-42290eff878d None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:44.358840 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee3f07d-702b-494e-84b9-42290eff878d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:34:44.359132 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee3f07d-702b-494e-84b9-42290eff878d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:44.360550 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee3f07d-702b-494e-84b9-42290eff878d None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:44.371568 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 621/1266] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:34:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:34:44.377847 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4592a48-d278-45ff-ba26-514669c0f97c None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:44.477957 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4592a48-d278-45ff-ba26-514669c0f97c None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:44.497577 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:44.510849 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2484c4-a610-4705-84df-0b479e9b31ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:44.511446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2484c4-a610-4705-84df-0b479e9b31ef None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:44.511728 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2484c4-a610-4705-84df-0b479e9b31ef None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:44.512040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2484c4-a610-4705-84df-0b479e9b31ef None admin] PATH_INFO: `/v3/users/placement` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:44.519673 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2484c4-a610-4705-84df-0b479e9b31ef None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:44.521131 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2484c4-a610-4705-84df-0b479e9b31ef None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:44.532945 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-7f2484c4-a610-4705-84df-0b479e9b31ef None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 03 12:34:44.532945 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:34:44.532945 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:34:44.532945 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:34:44.532945 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:34:44.532945 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:34:44.532945 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:34:44.532945 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:34:44.532945 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:34:44.532945 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:34:44.532945 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:34:44.532945 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:34:44.532945 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 12:34:44.532945 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 12:34:44.532945 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 12:34:44.532945 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 12:34:44.534102 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:34:44.534102 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:34:44.534102 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 12:34:44.534102 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:34:44.534102 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 12:34:44.534102 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:34:44.534102 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:34:44.534102 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:34:44.534102 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:34:44.534102 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:34:44.534102 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:34:44.534102 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:34:44.534102 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:34:44.534102 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:34:44.534102 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:34:44.534102 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:34:44.534102 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:34:44.534102 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:34:44.535453 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:34:44.535453 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:34:44.535453 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 12:34:44.535453 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 12:34:44.535453 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 12:34:44.535453 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 12:34:44.535453 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 12:34:44.535453 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 12:34:44.535453 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 03 12:34:44.535453 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:34:44.535453 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 646/1267] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:34:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:34:44.540097 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3f07d-702b-494e-84b9-42290eff878d None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:44.639862 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3f07d-702b-494e-84b9-42290eff878d None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:44.672973 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3388ffb-3642-4b83-8512-eddc8faa48ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:44.673581 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3388ffb-3642-4b83-8512-eddc8faa48ca None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:44.673857 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3388ffb-3642-4b83-8512-eddc8faa48ca None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:44.674124 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3388ffb-3642-4b83-8512-eddc8faa48ca None admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:44.674846 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3388ffb-3642-4b83-8512-eddc8faa48ca None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:34:44.675139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3388ffb-3642-4b83-8512-eddc8faa48ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:44.676582 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3388ffb-3642-4b83-8512-eddc8faa48ca None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:44.686251 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:44.694335 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 622/1268] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Oct 3 12:34:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:34:44.700369 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2484c4-a610-4705-84df-0b479e9b31ef None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:44.800254 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2484c4-a610-4705-84df-0b479e9b31ef None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:44.832831 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c4188e-b69d-4011-b3d9-22ff539dd1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:44.833460 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4188e-b69d-4011-b3d9-22ff539dd1ce None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:44.833741 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4188e-b69d-4011-b3d9-22ff539dd1ce None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:44.834013 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4188e-b69d-4011-b3d9-22ff539dd1ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:44.845166 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c4188e-b69d-4011-b3d9-22ff539dd1ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:44.846588 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c4188e-b69d-4011-b3d9-22ff539dd1ce None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:44.861646 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-06c4188e-b69d-4011-b3d9-22ff539dd1ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:34:44.861646 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:34:44.861646 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:34:44.861646 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:34:44.861646 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:34:44.861646 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:34:44.861646 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:34:44.861646 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:34:44.861646 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:34:44.861646 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:34:44.861646 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:34:44.861646 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:34:44.861646 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:34:44.861646 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:34:44.861646 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:34:44.861646 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:34:44.863031 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:34:44.863031 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:34:44.863031 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:34:44.863031 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:34:44.863031 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:34:44.863031 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:34:44.863031 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:34:44.863031 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:34:44.863031 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:34:44.863031 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:34:44.863031 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:34:44.863031 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:34:44.863031 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:34:44.863031 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:34:44.863031 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:34:44.863031 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:34:44.863031 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:34:44.864053 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:34:44.864053 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:34:44.864053 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:34:44.864053 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 647/1269] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:34:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:34:44.868754 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3388ffb-3642-4b83-8512-eddc8faa48ca None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:44.967573 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3388ffb-3642-4b83-8512-eddc8faa48ca None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:45.001328 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a981e-9005-4dc9-8b52-02335abe6340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:45.001931 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a981e-9005-4dc9-8b52-02335abe6340 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:45.002214 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a981e-9005-4dc9-8b52-02335abe6340 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:45.002489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a981e-9005-4dc9-8b52-02335abe6340 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:45.003149 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171a981e-9005-4dc9-8b52-02335abe6340 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:34:45.003482 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171a981e-9005-4dc9-8b52-02335abe6340 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:45.004843 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171a981e-9005-4dc9-8b52-02335abe6340 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:45.016019 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 623/1270] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:34:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:34:45.021873 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c4188e-b69d-4011-b3d9-22ff539dd1ce None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:45.057728 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:45.126328 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c4188e-b69d-4011-b3d9-22ff539dd1ce None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:45.159015 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1f982-4438-41f9-ac3c-8320ef18b082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:45.159669 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1f982-4438-41f9-ac3c-8320ef18b082 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:45.159945 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1f982-4438-41f9-ac3c-8320ef18b082 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:45.160218 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1f982-4438-41f9-ac3c-8320ef18b082 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:45.166476 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e1f982-4438-41f9-ac3c-8320ef18b082 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:45.167925 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e1f982-4438-41f9-ac3c-8320ef18b082 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:45.179434 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-f7e1f982-4438-41f9-ac3c-8320ef18b082 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:34:45.179434 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:34:45.179434 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:34:45.179434 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:34:45.179434 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:34:45.179434 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:34:45.179434 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:34:45.179434 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:34:45.179434 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:34:45.179434 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:34:45.179434 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:34:45.179434 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:34:45.179434 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 12:34:45.179434 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 12:34:45.179434 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 12:34:45.180441 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 12:34:45.180441 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:34:45.180441 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:34:45.180441 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:34:45.180441 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:34:45.180441 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:34:45.180441 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:34:45.180441 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:34:45.180441 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:34:45.180441 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:34:45.180441 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:34:45.180441 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:34:45.180441 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:34:45.180441 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:34:45.180441 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:34:45.180441 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:34:45.180441 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:34:45.181503 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 12:34:45.181503 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 12:34:45.181503 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 12:34:45.181503 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 12:34:45.181503 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 12:34:45.181503 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 12:34:45.181503 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:34:45.181503 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:34:45.181503 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 648/1271] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:34:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:34:45.186582 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a981e-9005-4dc9-8b52-02335abe6340 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:45.245204 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:45.287979 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a981e-9005-4dc9-8b52-02335abe6340 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:45.320906 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ebb5c-1bec-4ea0-be09-1bf0f3851514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:45.321445 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ebb5c-1bec-4ea0-be09-1bf0f3851514 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:45.321724 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ebb5c-1bec-4ea0-be09-1bf0f3851514 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:45.321998 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ebb5c-1bec-4ea0-be09-1bf0f3851514 None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:45.322684 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6ebb5c-1bec-4ea0-be09-1bf0f3851514 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:34:45.322987 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6ebb5c-1bec-4ea0-be09-1bf0f3851514 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:45.326437 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6ebb5c-1bec-4ea0-be09-1bf0f3851514 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:45.344727 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 624/1272] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 12:34:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:34:45.351058 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1f982-4438-41f9-ac3c-8320ef18b082 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:45.442138 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:45.450356 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1f982-4438-41f9-ac3c-8320ef18b082 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:45.484015 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f65d8f4-27b0-42e8-be90-7275ddcc5c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:45.484666 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65d8f4-27b0-42e8-be90-7275ddcc5c7e None admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:45.484945 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65d8f4-27b0-42e8-be90-7275ddcc5c7e None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:45.485218 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65d8f4-27b0-42e8-be90-7275ddcc5c7e None admin] PATH_INFO: `/v3/projects/4ea2fa8ad01240f7859182ebbe3051c7/users/25eef64e92e9422ea62508c3eb0a197e/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:45.487073 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f65d8f4-27b0-42e8-be90-7275ddcc5c7e None admin] RBAC: Authorizing `identity:create_grant(project_id=4ea2fa8ad01240f7859182ebbe3051c7, user_id=25eef64e92e9422ea62508c3eb0a197e, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:45.488509 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f65d8f4-27b0-42e8-be90-7275ddcc5c7e None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:45.582106 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 649/1273] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 12:34:45 2022] PUT /identity/v3/projects/4ea2fa8ad01240f7859182ebbe3051c7/users/25eef64e92e9422ea62508c3eb0a197e/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:34:47.715469 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e1e7d-e806-4d96-a9e7-c3649cca318c None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:47.716042 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e1e7d-e806-4d96-a9e7-c3649cca318c None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:47.716042 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e1e7d-e806-4d96-a9e7-c3649cca318c None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:47.717423 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 625/1274] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:34:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:34:47.722993 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a432e7-d617-4b3a-961f-3f0af5dd183d None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:47.723423 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a432e7-d617-4b3a-961f-3f0af5dd183d None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:47.723622 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a432e7-d617-4b3a-961f-3f0af5dd183d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:47.742675 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-b5a432e7-d617-4b3a-961f-3f0af5dd183d None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:34:47.743903 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-b5a432e7-d617-4b3a-961f-3f0af5dd183d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:34:47.808190 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 650/1275] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:34:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:34:48.083096 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e1e7d-e806-4d96-a9e7-c3649cca318c None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:48.184331 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e1e7d-e806-4d96-a9e7-c3649cca318c None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:48.202490 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:48.218625 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a230b72-e6fe-46c5-b162-81c06aa25f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:48.219417 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a230b72-e6fe-46c5-b162-81c06aa25f97 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:48.219585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a230b72-e6fe-46c5-b162-81c06aa25f97 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:48.219838 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a230b72-e6fe-46c5-b162-81c06aa25f97 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:48.229902 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a230b72-e6fe-46c5-b162-81c06aa25f97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 12:34:48.234982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-4a230b72-e6fe-46c5-b162-81c06aa25f97 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:34:48.234982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:34:48.234982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:34:48.234982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:34:48.234982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:34:48.234982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:34:48.234982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:34:48.234982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:34:48.234982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:34:48.234982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:34:48.234982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:34:48.234982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:34:48.234982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 12:34:48.234982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 12:34:48.234982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 12:34:48.234982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 12:34:48.236053 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 12:34:48.236053 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 12:34:48.236053 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 12:34:48.236053 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 12:34:48.236053 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:34:48.236053 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:34:48.236053 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:34:48.236053 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:34:48.236053 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:34:48.236053 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:34:48.236053 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:34:48.236053 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:34:48.236053 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:34:48.236053 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:34:48.236053 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:34:48.236053 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:34:48.236965 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:34:48.236965 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:34:48.236965 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:34:48.236965 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:34:48.236965 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 12:34:48.236965 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 12:34:48.236965 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 12:34:48.236965 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 12:34:48.236965 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 12:34:48.236965 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 12:34:48.236965 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 12:34:48.236965 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:34:48.236965 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 626/1276] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 12:34:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:34:48.242191 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a432e7-d617-4b3a-961f-3f0af5dd183d None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:48.321646 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:48.340333 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a432e7-d617-4b3a-961f-3f0af5dd183d None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:48.372893 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dae91c9-d1df-4ee1-a2e7-69f60d3ba36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:48.374007 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae91c9-d1df-4ee1-a2e7-69f60d3ba36f None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:48.374273 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae91c9-d1df-4ee1-a2e7-69f60d3ba36f None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:48.374545 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae91c9-d1df-4ee1-a2e7-69f60d3ba36f None admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:48.375200 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dae91c9-d1df-4ee1-a2e7-69f60d3ba36f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:34:48.375577 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dae91c9-d1df-4ee1-a2e7-69f60d3ba36f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:48.381355 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dae91c9-d1df-4ee1-a2e7-69f60d3ba36f None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:48.384682 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 651/1277] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 12:34:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:34:48.390842 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a230b72-e6fe-46c5-b162-81c06aa25f97 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:48.490606 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a230b72-e6fe-46c5-b162-81c06aa25f97 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:48.524175 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5913d68a-087d-4410-9cd0-0f25b6a28ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:48.524790 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913d68a-087d-4410-9cd0-0f25b6a28ace None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:48.525064 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913d68a-087d-4410-9cd0-0f25b6a28ace None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:48.525330 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913d68a-087d-4410-9cd0-0f25b6a28ace None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:48.535887 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5913d68a-087d-4410-9cd0-0f25b6a28ace None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:48.537366 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5913d68a-087d-4410-9cd0-0f25b6a28ace None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:48.551953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-5913d68a-087d-4410-9cd0-0f25b6a28ace None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:34:48.551953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:34:48.551953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:34:48.551953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:34:48.551953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:34:48.551953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:34:48.551953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:34:48.551953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:34:48.551953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:34:48.551953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:34:48.551953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:34:48.551953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:34:48.551953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:34:48.551953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:34:48.551953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:34:48.551953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:34:48.552956 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:34:48.552956 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:34:48.552956 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:34:48.552956 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:34:48.552956 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:34:48.552956 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:34:48.552956 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:34:48.552956 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:34:48.552956 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:34:48.552956 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:34:48.552956 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:34:48.552956 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:34:48.552956 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:34:48.552956 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:34:48.552956 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:34:48.552956 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:34:48.552956 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:34:48.554128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:34:48.554128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:34:48.554128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:34:48.554128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 627/1278] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:34:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:34:48.558959 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dae91c9-d1df-4ee1-a2e7-69f60d3ba36f None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:48.657937 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dae91c9-d1df-4ee1-a2e7-69f60d3ba36f None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:48.689875 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f65b09b-a91c-455b-a382-15d947338a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:48.690469 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f65b09b-a91c-455b-a382-15d947338a7a None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:48.690743 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f65b09b-a91c-455b-a382-15d947338a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:48.691008 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f65b09b-a91c-455b-a382-15d947338a7a None admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:48.691669 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f65b09b-a91c-455b-a382-15d947338a7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:34:48.692009 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f65b09b-a91c-455b-a382-15d947338a7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:48.693349 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f65b09b-a91c-455b-a382-15d947338a7a None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:48.705453 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 652/1279] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:34:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:34:48.711209 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5913d68a-087d-4410-9cd0-0f25b6a28ace None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:48.742030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:48.815032 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5913d68a-087d-4410-9cd0-0f25b6a28ace None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:48.850547 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493eb166-ffa5-44ed-83c1-95445a0f2591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:48.851471 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493eb166-ffa5-44ed-83c1-95445a0f2591 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:48.851749 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493eb166-ffa5-44ed-83c1-95445a0f2591 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:48.852021 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493eb166-ffa5-44ed-83c1-95445a0f2591 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:48.859343 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493eb166-ffa5-44ed-83c1-95445a0f2591 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:48.860771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493eb166-ffa5-44ed-83c1-95445a0f2591 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:48.872171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-493eb166-ffa5-44ed-83c1-95445a0f2591 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 03 12:34:48.872171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:34:48.872171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:34:48.872171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:34:48.872171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:34:48.872171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:34:48.872171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:34:48.872171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:34:48.872171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:34:48.872171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:34:48.872171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:34:48.872171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:34:48.872171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 12:34:48.872171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 12:34:48.872171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 12:34:48.872171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 12:34:48.873392 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:34:48.873392 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:34:48.873392 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 12:34:48.873392 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:34:48.873392 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 12:34:48.873392 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 12:34:48.873392 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:34:48.873392 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:34:48.873392 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:34:48.873392 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:34:48.873392 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:34:48.873392 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:34:48.873392 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:34:48.873392 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:34:48.873392 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:34:48.873392 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:34:48.873392 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:34:48.873392 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:34:48.875055 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:34:48.875055 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:34:48.875055 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 12:34:48.875055 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 12:34:48.875055 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 12:34:48.875055 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 12:34:48.875055 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 12:34:48.875055 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 12:34:48.875055 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 03 12:34:48.875055 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:34:48.875055 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 628/1280] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:34:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:34:48.879680 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f65b09b-a91c-455b-a382-15d947338a7a None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:48.883570 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:48.979416 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f65b09b-a91c-455b-a382-15d947338a7a None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:49.018071 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccea8375-2e89-46c8-81aa-12a5cf3eff76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:49.018681 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccea8375-2e89-46c8-81aa-12a5cf3eff76 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:49.018973 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccea8375-2e89-46c8-81aa-12a5cf3eff76 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:49.019227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccea8375-2e89-46c8-81aa-12a5cf3eff76 None admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:49.019981 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccea8375-2e89-46c8-81aa-12a5cf3eff76 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:34:49.020284 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccea8375-2e89-46c8-81aa-12a5cf3eff76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:49.021714 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccea8375-2e89-46c8-81aa-12a5cf3eff76 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:49.038517 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 653/1281] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Oct 3 12:34:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:34:49.044479 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493eb166-ffa5-44ed-83c1-95445a0f2591 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:49.134587 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:49.144511 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493eb166-ffa5-44ed-83c1-95445a0f2591 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:49.177746 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71164224-4f70-4786-b35c-b622b144b805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:49.178347 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71164224-4f70-4786-b35c-b622b144b805 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:49.178621 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71164224-4f70-4786-b35c-b622b144b805 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:49.178889 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71164224-4f70-4786-b35c-b622b144b805 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:49.250565 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71164224-4f70-4786-b35c-b622b144b805 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:49.252531 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71164224-4f70-4786-b35c-b622b144b805 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:49.354378 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:49.371418 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 629/1282] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 12:34:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 12:34:49.375104 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccea8375-2e89-46c8-81aa-12a5cf3eff76 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:49.443413 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:49.485715 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccea8375-2e89-46c8-81aa-12a5cf3eff76 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:49.522570 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b2b2d-cddb-4e89-a1f6-7abc99d4a5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:49.523142 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b2b2d-cddb-4e89-a1f6-7abc99d4a5d9 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:49.523456 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b2b2d-cddb-4e89-a1f6-7abc99d4a5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:49.523732 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b2b2d-cddb-4e89-a1f6-7abc99d4a5d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:49.535292 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67b2b2d-cddb-4e89-a1f6-7abc99d4a5d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:49.536756 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67b2b2d-cddb-4e89-a1f6-7abc99d4a5d9 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:49.552489 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-b67b2b2d-cddb-4e89-a1f6-7abc99d4a5d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:34:49.552489 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:34:49.552489 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:34:49.552489 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:34:49.552489 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:34:49.552489 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:34:49.552489 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:34:49.552489 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:34:49.552489 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:34:49.552489 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:34:49.552489 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:34:49.552489 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:34:49.552489 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 12:34:49.552489 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 12:34:49.552489 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 12:34:49.552489 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 12:34:49.553606 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:34:49.553606 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:34:49.553606 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:34:49.553606 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:34:49.553606 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:34:49.553606 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:34:49.553606 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:34:49.553606 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:34:49.553606 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:34:49.553606 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:34:49.553606 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:34:49.553606 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:34:49.553606 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:34:49.553606 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:34:49.553606 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:34:49.553606 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:34:49.553606 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 12:34:49.554732 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 12:34:49.554732 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 12:34:49.554732 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:34:49.554732 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 654/1283] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:34:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:34:49.559632 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71164224-4f70-4786-b35c-b622b144b805 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:49.664302 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71164224-4f70-4786-b35c-b622b144b805 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:49.702272 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd617db-b971-4ad6-bcba-f74b79fcbd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:49.702880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd617db-b971-4ad6-bcba-f74b79fcbd70 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:49.703158 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd617db-b971-4ad6-bcba-f74b79fcbd70 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:49.703471 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd617db-b971-4ad6-bcba-f74b79fcbd70 None admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:49.704104 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd617db-b971-4ad6-bcba-f74b79fcbd70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:34:49.704402 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd617db-b971-4ad6-bcba-f74b79fcbd70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:49.705761 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd617db-b971-4ad6-bcba-f74b79fcbd70 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:49.716839 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 630/1284] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 12:34:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:34:49.723222 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b2b2d-cddb-4e89-a1f6-7abc99d4a5d9 None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:49.823022 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b2b2d-cddb-4e89-a1f6-7abc99d4a5d9 None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:49.836926 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:49.856654 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e5dbb-e72e-4ae0-9dd9-12435b90562e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:49.857320 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e5dbb-e72e-4ae0-9dd9-12435b90562e None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:49.857559 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e5dbb-e72e-4ae0-9dd9-12435b90562e None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:49.857846 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e5dbb-e72e-4ae0-9dd9-12435b90562e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:49.864245 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594e5dbb-e72e-4ae0-9dd9-12435b90562e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:49.865668 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594e5dbb-e72e-4ae0-9dd9-12435b90562e None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:49.877348 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application [None req-594e5dbb-e72e-4ae0-9dd9-12435b90562e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:34:49.877348 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:34:49.877348 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:34:49.877348 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:34:49.877348 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:34:49.877348 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:34:49.877348 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:34:49.877348 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:34:49.877348 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:34:49.877348 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:34:49.877348 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:34:49.877348 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:34:49.877348 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 12:34:49.877348 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 12:34:49.877348 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 12:34:49.878427 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 12:34:49.878427 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:34:49.878427 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:34:49.878427 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:34:49.878427 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:34:49.878427 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:34:49.878427 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:34:49.878427 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:34:49.878427 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application with Lock( Oct 03 12:34:49.878427 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:34:49.878427 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:34:49.878427 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:34:49.878427 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:34:49.878427 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:34:49.878427 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:34:49.878427 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:34:49.878427 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:34:49.879365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 12:34:49.879365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 12:34:49.879365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 12:34:49.879365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 12:34:49.879365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 12:34:49.879365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 12:34:49.879365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 12:34:49.879365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: ERROR keystone.server.flask.application Oct 03 12:34:49.879365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 655/1285] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 12:34:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:34:49.884551 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd617db-b971-4ad6-bcba-f74b79fcbd70 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:49.934474 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:49.984430 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd617db-b971-4ad6-bcba-f74b79fcbd70 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:50.017301 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d701f4b-3a83-447f-86a5-ad655cb64c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:50.018438 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d701f4b-3a83-447f-86a5-ad655cb64c42 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:50.018438 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d701f4b-3a83-447f-86a5-ad655cb64c42 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:50.018438 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d701f4b-3a83-447f-86a5-ad655cb64c42 None admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:50.019302 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d701f4b-3a83-447f-86a5-ad655cb64c42 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:34:50.019426 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d701f4b-3a83-447f-86a5-ad655cb64c42 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:50.020846 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d701f4b-3a83-447f-86a5-ad655cb64c42 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:50.033322 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 631/1286] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 12:34:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:34:50.039680 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e5dbb-e72e-4ae0-9dd9-12435b90562e None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:50.141017 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e5dbb-e72e-4ae0-9dd9-12435b90562e None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:50.173700 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a969b-a7c7-4dae-820f-235c7f2f86a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:50.174287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a969b-a7c7-4dae-820f-235c7f2f86a1 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:50.174563 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a969b-a7c7-4dae-820f-235c7f2f86a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:50.174942 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a969b-a7c7-4dae-820f-235c7f2f86a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:50.175731 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a969b-a7c7-4dae-820f-235c7f2f86a1 None admin] RBAC: Adding query filter params (role.id=7933c45dcdd54ef0a98018ba8a8bddaa, scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7, user.id=25eef64e92e9422ea62508c3eb0a197e) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:34:50.176037 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a969b-a7c7-4dae-820f-235c7f2f86a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:50.181868 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a969b-a7c7-4dae-820f-235c7f2f86a1 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:50.183142 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 656/1287] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 12:34:50 2022] GET /identity/v3/role_assignments?user.id=25eef64e92e9422ea62508c3eb0a197e&scope.project.id=4ea2fa8ad01240f7859182ebbe3051c7&role.id=7933c45dcdd54ef0a98018ba8a8bddaa => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:34:52.378969 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484a7e1-a778-4c04-9f21-5740c2c370ba None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:52.379600 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484a7e1-a778-4c04-9f21-5740c2c370ba None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:52.379600 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484a7e1-a778-4c04-9f21-5740c2c370ba None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:52.380911 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 632/1288] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:34:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:34:52.386056 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9db504-02e1-4a5b-82fe-25eeef48e374 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:52.386449 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9db504-02e1-4a5b-82fe-25eeef48e374 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:52.386645 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9db504-02e1-4a5b-82fe-25eeef48e374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:52.405178 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-9d9db504-02e1-4a5b-82fe-25eeef48e374 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:34:52.406997 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-9d9db504-02e1-4a5b-82fe-25eeef48e374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:34:52.435482 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:52.473999 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 657/1289] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:34:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:34:52.744611 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484a7e1-a778-4c04-9f21-5740c2c370ba None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:52.845500 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484a7e1-a778-4c04-9f21-5740c2c370ba None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:52.856274 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:52.879618 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d33ec-6545-465a-8737-a13f248abaea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:52.880238 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d33ec-6545-465a-8737-a13f248abaea None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:52.880516 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d33ec-6545-465a-8737-a13f248abaea None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:52.880791 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d33ec-6545-465a-8737-a13f248abaea None admin] PATH_INFO: `/v3/services/placement` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:52.881368 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43d33ec-6545-465a-8737-a13f248abaea None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:52.882718 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43d33ec-6545-465a-8737-a13f248abaea None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:52.891880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-e43d33ec-6545-465a-8737-a13f248abaea None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 03 12:34:52.891880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:34:52.891880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 12:34:52.891880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 12:34:52.891880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 12:34:52.891880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 12:34:52.891880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 12:34:52.891880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:34:52.891880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 03 12:34:52.891880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:34:52.891880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 12:34:52.891880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:34:52.891880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:34:52.891880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:34:52.891880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:34:52.891880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:34:52.891880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:34:52.892910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:34:52.892910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:34:52.892910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:34:52.892910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:34:52.892910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:34:52.892910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:34:52.892910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 12:34:52.892910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 12:34:52.892910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 12:34:52.892910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 12:34:52.892910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:34:52.892910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:34:52.892910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:34:52.892910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:34:52.892910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:34:52.892910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:34:52.893747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:34:52.893747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:34:52.893747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:34:52.893747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:34:52.893747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:34:52.893747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:34:52.893747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:34:52.893747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:34:52.893747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:34:52.893747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:34:52.893747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 12:34:52.893747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:34:52.893747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 03 12:34:52.893747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:34:52.893747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 633/1290] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Oct 3 12:34:52 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:34:52.898908 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9db504-02e1-4a5b-82fe-25eeef48e374 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:52.994000 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9db504-02e1-4a5b-82fe-25eeef48e374 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:53.027212 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c053f8-a3f3-41fc-965a-2a13f4da4155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:53.028000 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c053f8-a3f3-41fc-965a-2a13f4da4155 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:53.028284 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c053f8-a3f3-41fc-965a-2a13f4da4155 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:53.028558 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c053f8-a3f3-41fc-965a-2a13f4da4155 None admin] PATH_INFO: `/v3/services` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:53.029186 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c053f8-a3f3-41fc-965a-2a13f4da4155 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:34:53.029494 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c053f8-a3f3-41fc-965a-2a13f4da4155 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:53.031547 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c053f8-a3f3-41fc-965a-2a13f4da4155 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:53.036288 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 658/1291] 10.222.0.33 () {56 vars in 1116 bytes} [Mon Oct 3 12:34:52 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:34:53.042067 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d33ec-6545-465a-8737-a13f248abaea None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:53.141741 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d33ec-6545-465a-8737-a13f248abaea None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:53.175340 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71382538-2da2-47e5-8c97-6a32c625038b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:53.175960 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71382538-2da2-47e5-8c97-6a32c625038b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:53.176238 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71382538-2da2-47e5-8c97-6a32c625038b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:53.176509 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71382538-2da2-47e5-8c97-6a32c625038b None admin] PATH_INFO: `/v3/services` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:53.177133 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71382538-2da2-47e5-8c97-6a32c625038b None admin] RBAC: Adding query filter params (type=placement) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:34:53.177432 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71382538-2da2-47e5-8c97-6a32c625038b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:53.178911 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71382538-2da2-47e5-8c97-6a32c625038b None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:53.185788 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 634/1292] 10.222.0.33 () {56 vars in 1116 bytes} [Mon Oct 3 12:34:53 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:34:55.338374 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fcbb92-b916-4f6a-a390-91b92c93f043 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:55.338929 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fcbb92-b916-4f6a-a390-91b92c93f043 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:55.338929 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fcbb92-b916-4f6a-a390-91b92c93f043 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:55.340278 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 659/1293] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:34:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:34:55.346435 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba810c2-53da-45ae-998a-a929b5076a68 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:55.346872 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba810c2-53da-45ae-998a-a929b5076a68 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:55.346995 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba810c2-53da-45ae-998a-a929b5076a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:55.366205 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-bba810c2-53da-45ae-998a-a929b5076a68 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:34:55.367534 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-bba810c2-53da-45ae-998a-a929b5076a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:34:55.397447 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:55.434068 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 635/1294] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:34:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:34:55.703916 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fcbb92-b916-4f6a-a390-91b92c93f043 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:55.735407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:55.803613 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fcbb92-b916-4f6a-a390-91b92c93f043 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:55.836189 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b65ee-7009-40e2-a52f-76e6062736a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:55.836843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b65ee-7009-40e2-a52f-76e6062736a4 None admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:55.837127 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b65ee-7009-40e2-a52f-76e6062736a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:55.837402 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b65ee-7009-40e2-a52f-76e6062736a4 None admin] PATH_INFO: `/v3/services` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:55.838049 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130b65ee-7009-40e2-a52f-76e6062736a4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:55.839416 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130b65ee-7009-40e2-a52f-76e6062736a4 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:55.919102 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 660/1295] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Oct 3 12:34:55 2022] POST /identity/v3/services => generated 246 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:34:58.103337 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a5939c-1b4e-4c6d-b3c9-328546640b03 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:58.103889 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a5939c-1b4e-4c6d-b3c9-328546640b03 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:58.103889 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a5939c-1b4e-4c6d-b3c9-328546640b03 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:58.105220 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 636/1296] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:34:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:34:58.111100 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e3b7f-5833-4924-8671-e62ab97b668e None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:58.111559 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e3b7f-5833-4924-8671-e62ab97b668e None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:58.111686 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e3b7f-5833-4924-8671-e62ab97b668e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:58.129965 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-688e3b7f-5833-4924-8671-e62ab97b668e None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:34:58.131143 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-688e3b7f-5833-4924-8671-e62ab97b668e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:34:58.202419 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 661/1297] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:34:58 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:34:58.488411 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a5939c-1b4e-4c6d-b3c9-328546640b03 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:58.586958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a5939c-1b4e-4c6d-b3c9-328546640b03 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:58.620272 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05d10e-cbac-4abb-b89f-f1b190e945b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:58.620922 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05d10e-cbac-4abb-b89f-f1b190e945b3 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:58.621197 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05d10e-cbac-4abb-b89f-f1b190e945b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:58.621464 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05d10e-cbac-4abb-b89f-f1b190e945b3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:58.622028 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df05d10e-cbac-4abb-b89f-f1b190e945b3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:58.623410 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df05d10e-cbac-4abb-b89f-f1b190e945b3 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:58.632835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-df05d10e-cbac-4abb-b89f-f1b190e945b3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 03 12:34:58.632835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:34:58.632835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 12:34:58.632835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 12:34:58.632835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 12:34:58.632835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 12:34:58.632835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 12:34:58.632835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:34:58.632835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 03 12:34:58.632835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:34:58.632835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 12:34:58.632835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:34:58.632835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:34:58.632835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:34:58.632835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:34:58.632835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:34:58.632835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:34:58.633832 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:34:58.633832 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:34:58.633832 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:34:58.633832 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:34:58.633832 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:34:58.633832 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:34:58.633832 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 12:34:58.633832 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 12:34:58.633832 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 12:34:58.633832 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 12:34:58.633832 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:34:58.633832 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:34:58.633832 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:34:58.633832 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:34:58.633832 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:34:58.633832 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:34:58.634742 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:34:58.634742 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:34:58.634742 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:34:58.634742 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:34:58.634742 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:34:58.634742 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:34:58.634742 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:34:58.634742 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:34:58.634742 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:34:58.634742 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:34:58.634742 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 12:34:58.634742 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:34:58.634742 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 03 12:34:58.634742 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:34:58.634742 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 637/1298] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Oct 3 12:34:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 12:34:58.640717 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688e3b7f-5833-4924-8671-e62ab97b668e None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:58.658687 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:58.741800 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688e3b7f-5833-4924-8671-e62ab97b668e None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:58.775089 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634f0ff-5b12-412d-85f1-a6214c2ecc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:58.775743 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634f0ff-5b12-412d-85f1-a6214c2ecc3a None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:58.776024 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634f0ff-5b12-412d-85f1-a6214c2ecc3a None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:58.776294 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634f0ff-5b12-412d-85f1-a6214c2ecc3a None admin] PATH_INFO: `/v3/services` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:58.776912 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a634f0ff-5b12-412d-85f1-a6214c2ecc3a None admin] RBAC: Adding query filter params (name=placement) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:34:58.777214 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a634f0ff-5b12-412d-85f1-a6214c2ecc3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:58.778562 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a634f0ff-5b12-412d-85f1-a6214c2ecc3a None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:58.783863 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 662/1299] 10.222.0.33 () {56 vars in 1116 bytes} [Mon Oct 3 12:34:58 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:34:58.789994 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05d10e-cbac-4abb-b89f-f1b190e945b3 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:58.821183 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:58.888516 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05d10e-cbac-4abb-b89f-f1b190e945b3 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:58.921658 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b46a6-3385-4985-b7c2-b462fc328c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:58.922257 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b46a6-3385-4985-b7c2-b462fc328c8b None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:58.922533 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b46a6-3385-4985-b7c2-b462fc328c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:58.922802 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b46a6-3385-4985-b7c2-b462fc328c8b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:58.923515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b46a6-3385-4985-b7c2-b462fc328c8b None admin] RBAC: Adding query filter params (interface=public, service_id=881abf360da4442b9a64439f7d631acd, region_id=RegionOne) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:34:58.923855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b46a6-3385-4985-b7c2-b462fc328c8b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:58.925205 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b46a6-3385-4985-b7c2-b462fc328c8b None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:58.930084 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 638/1300] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Oct 3 12:34:58 2022] GET /identity/v3/endpoints?service_id=881abf360da4442b9a64439f7d631acd&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:34:58.935745 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634f0ff-5b12-412d-85f1-a6214c2ecc3a None admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:34:59.020332 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:34:59.037251 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634f0ff-5b12-412d-85f1-a6214c2ecc3a None admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:34:59.070982 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25196f7-6572-45a8-a147-dc1a2bb8bfae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:34:59.071691 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25196f7-6572-45a8-a147-dc1a2bb8bfae None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:34:59.072007 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25196f7-6572-45a8-a147-dc1a2bb8bfae None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:34:59.072281 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25196f7-6572-45a8-a147-dc1a2bb8bfae None admin] PATH_INFO: `/v3/services` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:34:59.072881 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25196f7-6572-45a8-a147-dc1a2bb8bfae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:34:59.074229 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25196f7-6572-45a8-a147-dc1a2bb8bfae None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:34:59.081155 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 663/1301] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Oct 3 12:34:58 2022] GET /identity/v3/services => generated 2191 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:35:01.248441 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf5ffd5-3220-467f-972a-74b71511156e None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:35:01.248441 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf5ffd5-3220-467f-972a-74b71511156e None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:35:01.248441 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf5ffd5-3220-467f-972a-74b71511156e None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:35:01.250921 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 639/1302] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:35:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 12:35:01.256164 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79543f8-b447-4f3a-83ce-b9c7b0c9a8b3 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:35:01.256608 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79543f8-b447-4f3a-83ce-b9c7b0c9a8b3 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:35:01.256702 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79543f8-b447-4f3a-83ce-b9c7b0c9a8b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:35:01.275204 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-e79543f8-b447-4f3a-83ce-b9c7b0c9a8b3 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:35:01.276419 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-e79543f8-b447-4f3a-83ce-b9c7b0c9a8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:35:01.360240 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 664/1303] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:35:01 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 12:35:01.634485 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf5ffd5-3220-467f-972a-74b71511156e None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:35:01.747022 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:35:01.755860 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf5ffd5-3220-467f-972a-74b71511156e None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:35:01.793107 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8511ec69-3fdd-47df-8d43-fa00979dd757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:35:01.793730 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511ec69-3fdd-47df-8d43-fa00979dd757 None admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:35:01.794006 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511ec69-3fdd-47df-8d43-fa00979dd757 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:35:01.794274 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511ec69-3fdd-47df-8d43-fa00979dd757 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:35:01.794843 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8511ec69-3fdd-47df-8d43-fa00979dd757 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:35:01.796413 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8511ec69-3fdd-47df-8d43-fa00979dd757 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:35:01.805120 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application [None req-8511ec69-3fdd-47df-8d43-fa00979dd757 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 03 12:35:01.805120 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:35:01.805120 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 12:35:01.805120 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 12:35:01.805120 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 12:35:01.805120 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 12:35:01.805120 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 12:35:01.805120 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:35:01.805120 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 03 12:35:01.805120 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:35:01.805120 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 12:35:01.805120 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:35:01.805120 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 12:35:01.805120 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 12:35:01.805120 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 12:35:01.805120 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 12:35:01.805120 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 12:35:01.806323 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 12:35:01.806323 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 12:35:01.806323 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 12:35:01.806323 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 12:35:01.806323 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 12:35:01.806323 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 12:35:01.806323 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 12:35:01.806323 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 12:35:01.806323 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 12:35:01.806323 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 12:35:01.806323 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 12:35:01.806323 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 12:35:01.806323 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 12:35:01.806323 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 12:35:01.806323 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 12:35:01.806323 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 12:35:01.807576 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 12:35:01.807576 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application with Lock( Oct 03 12:35:01.807576 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 12:35:01.807576 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self._enter() Oct 03 12:35:01.807576 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 12:35:01.807576 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 12:35:01.807576 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 12:35:01.807576 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application return self.creator() Oct 03 12:35:01.807576 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 12:35:01.807576 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application created_value = creator( Oct 03 12:35:01.807576 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 12:35:01.807576 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 12:35:01.807576 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 03 12:35:01.807576 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: ERROR keystone.server.flask.application Oct 03 12:35:01.807576 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 640/1304] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Oct 3 12:35:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 12:35:01.812816 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79543f8-b447-4f3a-83ce-b9c7b0c9a8b3 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:35:01.886866 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:35:01.909781 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79543f8-b447-4f3a-83ce-b9c7b0c9a8b3 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:35:01.943491 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02964f84-111b-46b1-aaeb-485f30f487e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:35:01.944091 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02964f84-111b-46b1-aaeb-485f30f487e2 None admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:35:01.944371 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02964f84-111b-46b1-aaeb-485f30f487e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:35:01.944637 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02964f84-111b-46b1-aaeb-485f30f487e2 None admin] PATH_INFO: `/v3/services` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:35:01.945248 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02964f84-111b-46b1-aaeb-485f30f487e2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:35:01.945548 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02964f84-111b-46b1-aaeb-485f30f487e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:35:01.946874 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02964f84-111b-46b1-aaeb-485f30f487e2 None admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:35:01.951890 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 665/1305] 10.222.0.33 () {56 vars in 1116 bytes} [Mon Oct 3 12:35:01 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:35:01.958216 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8511ec69-3fdd-47df-8d43-fa00979dd757 None admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:35:02.056296 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8511ec69-3fdd-47df-8d43-fa00979dd757 None admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:35:02.090825 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc322c9-165d-4970-a78d-117fe1b4dd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:35:02.091456 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc322c9-165d-4970-a78d-117fe1b4dd65 None admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:35:02.091739 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc322c9-165d-4970-a78d-117fe1b4dd65 None admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:35:02.092005 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc322c9-165d-4970-a78d-117fe1b4dd65 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:35:02.092665 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc322c9-165d-4970-a78d-117fe1b4dd65 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:35:02.094274 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc322c9-165d-4970-a78d-117fe1b4dd65 None admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:35:02.195338 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 641/1306] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:35:01 2022] POST /identity/v3/endpoints => generated 331 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:35:11.948929 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=30 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:35:12.189286 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:40:56.227049 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2292fe4-d0b9-4af3-bdf2-0c129b867644 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:40:56.227673 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2292fe4-d0b9-4af3-bdf2-0c129b867644 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:40:56.227995 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2292fe4-d0b9-4af3-bdf2-0c129b867644 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:40:56.237619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd8a61-2bd6-482a-bdd2-da41652ce3fd None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:40:56.238149 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd8a61-2bd6-482a-bdd2-da41652ce3fd None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:40:56.238524 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd8a61-2bd6-482a-bdd2-da41652ce3fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:40:56.244883 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 666/1307] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:40:56 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:40:56.415840 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-60cd8a61-2bd6-482a-bdd2-da41652ce3fd None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:40:56.417066 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-60cd8a61-2bd6-482a-bdd2-da41652ce3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:40:56.614055 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 642/1308] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:40:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:40:56.919129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0687a261-61f2-4894-be7f-d50a8389564f None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:40:56.919917 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0687a261-61f2-4894-be7f-d50a8389564f None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:40:56.920453 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0687a261-61f2-4894-be7f-d50a8389564f None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:40:56.922420 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 667/1309] 10.222.0.33 () {54 vars in 964 bytes} [Mon Oct 3 12:40:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:40:56.930163 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b5be1-f72a-4f6d-a004-a2f113aeb346 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:40:56.930510 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b5be1-f72a-4f6d-a004-a2f113aeb346 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:40:56.930780 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b5be1-f72a-4f6d-a004-a2f113aeb346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:40:57.020607 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-da1b5be1-f72a-4f6d-a004-a2f113aeb346 None None] MFA Rules not processed for user `eb327a8a18ca414ab8b4111a4d13aad8`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:40:57.021936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-da1b5be1-f72a-4f6d-a004-a2f113aeb346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:40:57.044243 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:40:57.122690 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 643/1310] 10.222.0.33 () {58 vars in 1061 bytes} [Mon Oct 3 12:40:56 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:40:57.132692 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0687a261-61f2-4894-be7f-d50a8389564f None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:40:57.134440 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-0687a261-61f2-4894-be7f-d50a8389564f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:40:57.278319 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0687a261-61f2-4894-be7f-d50a8389564f None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:40:57.369606 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de896c-e1e8-4ef0-a4a3-b264ae4614b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:40:57.370516 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de896c-e1e8-4ef0-a4a3-b264ae4614b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:40:57.370948 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de896c-e1e8-4ef0-a4a3-b264ae4614b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:40:57.371388 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de896c-e1e8-4ef0-a4a3-b264ae4614b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:40:57.373131 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-a5de896c-e1e8-4ef0-a4a3-b264ae4614b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:40:57.435255 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:40:57.494073 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5de896c-e1e8-4ef0-a4a3-b264ae4614b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:40:57.495845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5de896c-e1e8-4ef0-a4a3-b264ae4614b9 service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:40:57.516211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 668/1311] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:40:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:40:59.996181 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce9d3d-5cdc-4575-a223-4497b023c73e None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:40:59.996181 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce9d3d-5cdc-4575-a223-4497b023c73e None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:40:59.996181 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce9d3d-5cdc-4575-a223-4497b023c73e None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:40:59.996735 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 644/1312] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:40:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:41:00.002152 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0fc171-bfcf-4310-81d3-f0bb6fc00c82 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:00.002565 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0fc171-bfcf-4310-81d3-f0bb6fc00c82 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:00.002796 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0fc171-bfcf-4310-81d3-f0bb6fc00c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:00.085337 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-7a0fc171-bfcf-4310-81d3-f0bb6fc00c82 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:41:00.086547 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-7a0fc171-bfcf-4310-81d3-f0bb6fc00c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:00.179838 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 669/1313] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:41:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:41:00.462408 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ce9d3d-5cdc-4575-a223-4497b023c73e None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:41:00.463961 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b4ce9d3d-5cdc-4575-a223-4497b023c73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:00.567053 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ce9d3d-5cdc-4575-a223-4497b023c73e None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:41:00.637174 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:41:00.639440 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028849ed-beb7-4b8f-a21d-a446f3b07e53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:41:00.640048 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028849ed-beb7-4b8f-a21d-a446f3b07e53 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:00.640332 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028849ed-beb7-4b8f-a21d-a446f3b07e53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:00.640603 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028849ed-beb7-4b8f-a21d-a446f3b07e53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:00.642747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-028849ed-beb7-4b8f-a21d-a446f3b07e53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:00.756384 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028849ed-beb7-4b8f-a21d-a446f3b07e53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:41:00.757804 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028849ed-beb7-4b8f-a21d-a446f3b07e53 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:41:00.775238 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 645/1314] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:41:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:41:03.517112 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9df452-119a-478c-a3f9-18feb15d5bf4 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:03.517644 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9df452-119a-478c-a3f9-18feb15d5bf4 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:03.517644 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9df452-119a-478c-a3f9-18feb15d5bf4 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:03.518913 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 670/1315] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 12:41:03.524048 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd79200-cca4-4bb7-a7f5-76ded4a779d3 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:03.524466 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd79200-cca4-4bb7-a7f5-76ded4a779d3 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:03.524651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd79200-cca4-4bb7-a7f5-76ded4a779d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:03.589852 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-7dd79200-cca4-4bb7-a7f5-76ded4a779d3 None None] MFA Rules not processed for user `b8101b669ca44866a11705e7d0f3b601`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:41:03.591052 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-7dd79200-cca4-4bb7-a7f5-76ded4a779d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:03.642502 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:41:03.685769 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 646/1316] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:41:03 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:41:03.997638 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc8514-6d88-4293-aa78-8e7dfa8dc0f4 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:03.997932 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc8514-6d88-4293-aa78-8e7dfa8dc0f4 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:03.998209 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc8514-6d88-4293-aa78-8e7dfa8dc0f4 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:03.999583 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 671/1317] 10.222.0.33 () {54 vars in 964 bytes} [Mon Oct 3 12:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:41:04.006197 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c382c-5dbd-4822-ab66-485f5a9a6fdd None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:04.006511 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c382c-5dbd-4822-ab66-485f5a9a6fdd None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:04.006787 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c382c-5dbd-4822-ab66-485f5a9a6fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:04.081461 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-013c382c-5dbd-4822-ab66-485f5a9a6fdd None None] MFA Rules not processed for user `eb327a8a18ca414ab8b4111a4d13aad8`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:41:04.082652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-013c382c-5dbd-4822-ab66-485f5a9a6fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:04.156510 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 647/1318] 10.222.0.33 () {58 vars in 1061 bytes} [Mon Oct 3 12:41:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:41:04.166524 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fc8514-6d88-4293-aa78-8e7dfa8dc0f4 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:41:04.168072 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-c6fc8514-6d88-4293-aa78-8e7dfa8dc0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:04.179013 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:41:04.275756 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fc8514-6d88-4293-aa78-8e7dfa8dc0f4 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:41:04.350978 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6988afdf-8fe5-49b9-93c6-dcdacd631e58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:41:04.351622 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988afdf-8fe5-49b9-93c6-dcdacd631e58 service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:04.351906 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988afdf-8fe5-49b9-93c6-dcdacd631e58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:04.352173 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988afdf-8fe5-49b9-93c6-dcdacd631e58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:04.353733 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-6988afdf-8fe5-49b9-93c6-dcdacd631e58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:04.414613 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:41:04.460066 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6988afdf-8fe5-49b9-93c6-dcdacd631e58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:41:04.461486 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6988afdf-8fe5-49b9-93c6-dcdacd631e58 service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:41:04.480251 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 672/1319] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:41:04 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:41:07.790069 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a84d7c5-8d3e-4459-a28e-e3c6b4f5287e None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:07.790596 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a84d7c5-8d3e-4459-a28e-e3c6b4f5287e None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:07.790596 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a84d7c5-8d3e-4459-a28e-e3c6b4f5287e None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:07.791978 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 648/1320] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:41:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:41:07.797278 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0fd616-1de1-428d-82c7-b78e96b25ae2 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:07.797695 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0fd616-1de1-428d-82c7-b78e96b25ae2 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:07.797860 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0fd616-1de1-428d-82c7-b78e96b25ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:07.863067 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-2e0fd616-1de1-428d-82c7-b78e96b25ae2 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:41:07.864286 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-2e0fd616-1de1-428d-82c7-b78e96b25ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:07.960714 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 673/1321] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:41:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:41:08.249312 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a84d7c5-8d3e-4459-a28e-e3c6b4f5287e None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:41:08.250958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-6a84d7c5-8d3e-4459-a28e-e3c6b4f5287e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:08.301448 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:41:08.355742 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a84d7c5-8d3e-4459-a28e-e3c6b4f5287e None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:41:08.427891 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4295eb63-fade-49ea-9c70-191e7fe6e4c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:41:08.428502 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295eb63-fade-49ea-9c70-191e7fe6e4c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:08.428779 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295eb63-fade-49ea-9c70-191e7fe6e4c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:08.429049 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295eb63-fade-49ea-9c70-191e7fe6e4c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:08.430751 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4295eb63-fade-49ea-9c70-191e7fe6e4c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:08.528975 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:41:08.542700 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4295eb63-fade-49ea-9c70-191e7fe6e4c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:41:08.544152 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4295eb63-fade-49ea-9c70-191e7fe6e4c6 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:41:08.563034 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 649/1322] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:41:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:41:11.705307 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b9488-6071-44f8-afac-01be753bd192 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:11.705846 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b9488-6071-44f8-afac-01be753bd192 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:11.705964 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b9488-6071-44f8-afac-01be753bd192 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:11.707330 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 674/1323] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:41:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:41:11.712589 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355be7c-16fc-4cd7-bed4-75fa99689dd9 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:11.712978 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355be7c-16fc-4cd7-bed4-75fa99689dd9 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:11.713145 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355be7c-16fc-4cd7-bed4-75fa99689dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:11.778899 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-5355be7c-16fc-4cd7-bed4-75fa99689dd9 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:41:11.780125 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-5355be7c-16fc-4cd7-bed4-75fa99689dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:11.876844 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 650/1324] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:41:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:41:12.165588 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b9488-6071-44f8-afac-01be753bd192 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:41:12.167121 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-863b9488-6071-44f8-afac-01be753bd192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:12.190071 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:41:12.276954 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b9488-6071-44f8-afac-01be753bd192 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:41:12.354061 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a3e96d-e01f-495e-989b-01db84b2e765 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:41:12.354844 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3e96d-e01f-495e-989b-01db84b2e765 service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:12.355123 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3e96d-e01f-495e-989b-01db84b2e765 service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:12.355431 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3e96d-e01f-495e-989b-01db84b2e765 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:12.357017 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-37a3e96d-e01f-495e-989b-01db84b2e765 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:12.429158 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:41:12.466309 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a3e96d-e01f-495e-989b-01db84b2e765 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:41:12.467794 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a3e96d-e01f-495e-989b-01db84b2e765 service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:41:12.487539 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 675/1325] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:41:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:41:14.686791 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf4cae-79dd-42e5-963f-0e2ff87877bc None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:14.687378 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf4cae-79dd-42e5-963f-0e2ff87877bc None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:14.687378 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf4cae-79dd-42e5-963f-0e2ff87877bc None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:14.688929 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 651/1326] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:41:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:41:14.693946 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61624653-3cd2-407a-9c0d-0c5df00745ea None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:14.694370 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61624653-3cd2-407a-9c0d-0c5df00745ea None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:14.694482 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61624653-3cd2-407a-9c0d-0c5df00745ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:14.760913 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-61624653-3cd2-407a-9c0d-0c5df00745ea None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:41:14.762096 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-61624653-3cd2-407a-9c0d-0c5df00745ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:14.857437 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 676/1327] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:41:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:41:15.161750 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf4cae-79dd-42e5-963f-0e2ff87877bc None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:41:15.163348 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-bcbf4cae-79dd-42e5-963f-0e2ff87877bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:15.229029 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:41:15.275217 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf4cae-79dd-42e5-963f-0e2ff87877bc None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:41:15.362668 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0936371-80e9-4bc8-ab61-3bd431a1165a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:41:15.363189 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0936371-80e9-4bc8-ab61-3bd431a1165a service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:15.363521 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0936371-80e9-4bc8-ab61-3bd431a1165a service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:15.363799 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0936371-80e9-4bc8-ab61-3bd431a1165a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:15.365379 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-f0936371-80e9-4bc8-ab61-3bd431a1165a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:15.478315 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:41:15.480752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0936371-80e9-4bc8-ab61-3bd431a1165a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:41:15.482193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0936371-80e9-4bc8-ab61-3bd431a1165a service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:41:15.502097 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 652/1328] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:41:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:41:17.755785 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d50ce-a696-497b-ade2-d8598fca1c85 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:17.756272 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d50ce-a696-497b-ade2-d8598fca1c85 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:17.756382 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d50ce-a696-497b-ade2-d8598fca1c85 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:17.758933 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 677/1329] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:41:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:41:17.764376 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b26dc4-bdbd-47ca-a980-32473ccc8278 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:17.767590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b26dc4-bdbd-47ca-a980-32473ccc8278 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:17.768018 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b26dc4-bdbd-47ca-a980-32473ccc8278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:17.841626 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-29b26dc4-bdbd-47ca-a980-32473ccc8278 None None] MFA Rules not processed for user `b8101b669ca44866a11705e7d0f3b601`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:41:17.842846 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-29b26dc4-bdbd-47ca-a980-32473ccc8278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:17.942489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 653/1330] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:41:17 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:41:18.237118 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d50ce-a696-497b-ade2-d8598fca1c85 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:41:18.238649 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-7a1d50ce-a696-497b-ade2-d8598fca1c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:18.272005 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:41:18.352091 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d50ce-a696-497b-ade2-d8598fca1c85 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:41:18.444734 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561204bf-b340-49c6-ac88-08ac298f96a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:41:18.445355 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561204bf-b340-49c6-ac88-08ac298f96a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:18.445631 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561204bf-b340-49c6-ac88-08ac298f96a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:18.445899 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561204bf-b340-49c6-ac88-08ac298f96a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:18.447541 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-561204bf-b340-49c6-ac88-08ac298f96a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:18.530319 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:41:18.569291 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561204bf-b340-49c6-ac88-08ac298f96a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:41:18.570736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561204bf-b340-49c6-ac88-08ac298f96a5 service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:41:18.589943 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 678/1331] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:41:18 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:41:21.145102 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030fd47-c972-4786-b48f-bf8ee4383273 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:21.145698 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030fd47-c972-4786-b48f-bf8ee4383273 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:21.145698 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030fd47-c972-4786-b48f-bf8ee4383273 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:21.146965 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 654/1332] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:41:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:41:21.152127 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f18453-f674-4568-93d2-01989ea4dedb None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:21.152530 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f18453-f674-4568-93d2-01989ea4dedb None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:21.152670 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f18453-f674-4568-93d2-01989ea4dedb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:21.219500 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-06f18453-f674-4568-93d2-01989ea4dedb None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:41:21.221738 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-06f18453-f674-4568-93d2-01989ea4dedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:21.316615 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 679/1333] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:41:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:41:21.628630 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b030fd47-c972-4786-b48f-bf8ee4383273 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:41:21.630152 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b030fd47-c972-4786-b48f-bf8ee4383273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:21.699545 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:41:21.732761 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b030fd47-c972-4786-b48f-bf8ee4383273 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:41:21.808313 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6478a1c2-fe92-4366-9b57-8df4c8f1a3ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:41:21.808922 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478a1c2-fe92-4366-9b57-8df4c8f1a3ab service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:21.809198 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478a1c2-fe92-4366-9b57-8df4c8f1a3ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:21.809470 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478a1c2-fe92-4366-9b57-8df4c8f1a3ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:21.811057 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-6478a1c2-fe92-4366-9b57-8df4c8f1a3ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:21.917374 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6478a1c2-fe92-4366-9b57-8df4c8f1a3ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:41:21.918722 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6478a1c2-fe92-4366-9b57-8df4c8f1a3ab service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:41:21.929123 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:41:21.936851 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 655/1334] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:41:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:41:24.123580 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a2ce2-36ec-49e9-876e-402d09e3ea88 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:24.124144 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a2ce2-36ec-49e9-876e-402d09e3ea88 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:24.124144 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a2ce2-36ec-49e9-876e-402d09e3ea88 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:24.125450 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 680/1335] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:41:24.130625 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab483852-d62c-4954-86c1-e554c73ce4e5 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:24.131009 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab483852-d62c-4954-86c1-e554c73ce4e5 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:24.131179 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab483852-d62c-4954-86c1-e554c73ce4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:24.197077 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-ab483852-d62c-4954-86c1-e554c73ce4e5 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:41:24.198256 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ab483852-d62c-4954-86c1-e554c73ce4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:24.296194 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 656/1336] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:41:24.577502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a2ce2-36ec-49e9-876e-402d09e3ea88 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:41:24.579016 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-104a2ce2-36ec-49e9-876e-402d09e3ea88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:24.619080 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:41:24.682558 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a2ce2-36ec-49e9-876e-402d09e3ea88 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:41:24.753874 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f31da-42c7-411d-a413-86dc8e4e6760 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:41:24.754468 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f31da-42c7-411d-a413-86dc8e4e6760 service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:24.754742 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f31da-42c7-411d-a413-86dc8e4e6760 service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:24.755010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f31da-42c7-411d-a413-86dc8e4e6760 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:24.756597 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-cc7f31da-42c7-411d-a413-86dc8e4e6760 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:24.837886 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:41:24.863419 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f31da-42c7-411d-a413-86dc8e4e6760 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:41:24.864839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f31da-42c7-411d-a413-86dc8e4e6760 service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:41:24.883167 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 681/1337] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:41:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:41:27.619052 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f220f3-ac0a-4b80-9b6d-737ebeb99278 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:27.619661 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f220f3-ac0a-4b80-9b6d-737ebeb99278 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:27.619661 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f220f3-ac0a-4b80-9b6d-737ebeb99278 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:27.620887 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 657/1338] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:41:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:41:27.626045 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db351b53-5394-4c7c-86ad-45403401c4d2 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:27.626479 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db351b53-5394-4c7c-86ad-45403401c4d2 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:27.626585 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db351b53-5394-4c7c-86ad-45403401c4d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:27.691837 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-db351b53-5394-4c7c-86ad-45403401c4d2 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:41:27.693027 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-db351b53-5394-4c7c-86ad-45403401c4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:27.790021 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 682/1339] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:41:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:41:28.069240 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f220f3-ac0a-4b80-9b6d-737ebeb99278 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:41:28.070750 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-31f220f3-ac0a-4b80-9b6d-737ebeb99278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:28.145223 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:41:28.171668 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f220f3-ac0a-4b80-9b6d-737ebeb99278 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:41:28.247088 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e8d30-b37a-413b-a289-5f1d681a2c4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:41:28.247909 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e8d30-b37a-413b-a289-5f1d681a2c4c service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:28.248189 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e8d30-b37a-413b-a289-5f1d681a2c4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:28.248452 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e8d30-b37a-413b-a289-5f1d681a2c4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:28.250059 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4b5e8d30-b37a-413b-a289-5f1d681a2c4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:28.361426 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e8d30-b37a-413b-a289-5f1d681a2c4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:41:28.363474 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e8d30-b37a-413b-a289-5f1d681a2c4c service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:41:28.381681 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 658/1340] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:41:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:41:32.644836 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad01c4-8e0d-4ad2-8cee-a0acf2d0cfa5 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:32.645396 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad01c4-8e0d-4ad2-8cee-a0acf2d0cfa5 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:32.645396 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad01c4-8e0d-4ad2-8cee-a0acf2d0cfa5 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:32.646693 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 683/1341] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:41:32.652119 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afc3b8-706e-4b51-a0ce-7bc6a796767a None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:32.652527 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afc3b8-706e-4b51-a0ce-7bc6a796767a None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:32.652675 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afc3b8-706e-4b51-a0ce-7bc6a796767a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:32.658151 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:41:32.719968 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-12afc3b8-706e-4b51-a0ce-7bc6a796767a None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:41:32.721714 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-12afc3b8-706e-4b51-a0ce-7bc6a796767a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:32.818498 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 659/1342] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:41:33.113479 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad01c4-8e0d-4ad2-8cee-a0acf2d0cfa5 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:41:33.115070 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-78ad01c4-8e0d-4ad2-8cee-a0acf2d0cfa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:33.164175 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:41:33.222576 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad01c4-8e0d-4ad2-8cee-a0acf2d0cfa5 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:41:33.303260 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ec770-fda2-4ef2-a331-0520e0b7393a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:41:33.303900 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ec770-fda2-4ef2-a331-0520e0b7393a service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:33.304187 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ec770-fda2-4ef2-a331-0520e0b7393a service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:33.304453 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ec770-fda2-4ef2-a331-0520e0b7393a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:33.306033 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-096ec770-fda2-4ef2-a331-0520e0b7393a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:33.413337 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:41:33.429806 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ec770-fda2-4ef2-a331-0520e0b7393a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:41:33.431260 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ec770-fda2-4ef2-a331-0520e0b7393a service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:41:33.449862 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 684/1343] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:41:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:41:36.729445 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7814b-0286-4a4a-b992-16d73c48589f None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:36.730010 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7814b-0286-4a4a-b992-16d73c48589f None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:36.730103 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7814b-0286-4a4a-b992-16d73c48589f None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:36.731468 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 660/1344] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:41:36.736771 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25abe30-0511-4069-a594-0177777fce37 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:36.737242 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25abe30-0511-4069-a594-0177777fce37 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:36.737338 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25abe30-0511-4069-a594-0177777fce37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:36.802581 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-a25abe30-0511-4069-a594-0177777fce37 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:41:36.803800 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-a25abe30-0511-4069-a594-0177777fce37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:36.901394 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 685/1345] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:41:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:41:37.199408 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7814b-0286-4a4a-b992-16d73c48589f None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:41:37.200929 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d8e7814b-0286-4a4a-b992-16d73c48589f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:37.294088 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:41:37.310421 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7814b-0286-4a4a-b992-16d73c48589f None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:41:37.390487 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed99274f-c806-4854-a807-b10299f9cb3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:41:37.391112 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99274f-c806-4854-a807-b10299f9cb3d service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:37.391436 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99274f-c806-4854-a807-b10299f9cb3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:37.391751 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99274f-c806-4854-a807-b10299f9cb3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:37.394023 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ed99274f-c806-4854-a807-b10299f9cb3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:37.519812 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed99274f-c806-4854-a807-b10299f9cb3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:41:37.521271 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed99274f-c806-4854-a807-b10299f9cb3d service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:41:37.541523 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 661/1346] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:41:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:41:41.971989 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1bad2-a990-4d58-8240-fbddb38e7528 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:41.972555 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1bad2-a990-4d58-8240-fbddb38e7528 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:41.972555 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1bad2-a990-4d58-8240-fbddb38e7528 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:41.974168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 686/1347] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:41:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:41:41.980040 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61fc29-23f3-46ec-9417-eb366f003777 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:41.980503 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61fc29-23f3-46ec-9417-eb366f003777 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:41.980583 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61fc29-23f3-46ec-9417-eb366f003777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:41.997482 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:41:42.063859 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-3a61fc29-23f3-46ec-9417-eb366f003777 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:41:42.065119 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-3a61fc29-23f3-46ec-9417-eb366f003777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:42.187502 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 662/1348] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:41:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:41:42.475894 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce1bad2-a990-4d58-8240-fbddb38e7528 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:41:42.477422 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-0ce1bad2-a990-4d58-8240-fbddb38e7528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:42.535522 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:41:42.579917 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce1bad2-a990-4d58-8240-fbddb38e7528 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:41:42.652610 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd94db-fc6a-44c4-9c23-85c4d5ce8f28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:41:42.653226 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd94db-fc6a-44c4-9c23-85c4d5ce8f28 service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:42.653509 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd94db-fc6a-44c4-9c23-85c4d5ce8f28 service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:42.653784 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd94db-fc6a-44c4-9c23-85c4d5ce8f28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:42.655419 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-37cd94db-fc6a-44c4-9c23-85c4d5ce8f28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:42.764958 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:41:42.768946 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd94db-fc6a-44c4-9c23-85c4d5ce8f28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:41:42.770356 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd94db-fc6a-44c4-9c23-85c4d5ce8f28 service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:41:42.789139 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 687/1349] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:41:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:41:47.711579 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a3034a-82ba-496d-ad31-2a9020935349 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:47.711579 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a3034a-82ba-496d-ad31-2a9020935349 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:47.712072 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a3034a-82ba-496d-ad31-2a9020935349 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:47.713099 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 663/1350] 10.222.0.33 () {54 vars in 921 bytes} [Mon Oct 3 12:41:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:41:47.719019 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd8484-e058-4270-8e1b-cd136d16e951 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:47.719854 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd8484-e058-4270-8e1b-cd136d16e951 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:47.722618 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd8484-e058-4270-8e1b-cd136d16e951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:47.813681 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-4abd8484-e058-4270-8e1b-cd136d16e951 None None] MFA Rules not processed for user `25eef64e92e9422ea62508c3eb0a197e`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:41:47.815890 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-4abd8484-e058-4270-8e1b-cd136d16e951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:47.993566 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 688/1351] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Oct 3 12:41:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:41:52.180676 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=59 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:41:52.355774 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab3d33-f423-42fc-ac14-be942aa1778b None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:52.356055 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab3d33-f423-42fc-ac14-be942aa1778b None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:52.356522 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab3d33-f423-42fc-ac14-be942aa1778b None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:52.357868 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 664/1352] 10.222.0.33 () {54 vars in 921 bytes} [Mon Oct 3 12:41:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:41:52.364204 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5992f8d-4688-4d3e-8071-bc9c7cd126c3 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:52.364656 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5992f8d-4688-4d3e-8071-bc9c7cd126c3 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:52.364782 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5992f8d-4688-4d3e-8071-bc9c7cd126c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:52.430018 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:41:52.453098 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-d5992f8d-4688-4d3e-8071-bc9c7cd126c3 None None] MFA Rules not processed for user `25eef64e92e9422ea62508c3eb0a197e`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:41:52.454287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-d5992f8d-4688-4d3e-8071-bc9c7cd126c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:52.569562 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 689/1353] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Oct 3 12:41:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:41:53.624883 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9897d072-076c-48c2-8a78-6b1c29c6e2a0 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:53.625393 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9897d072-076c-48c2-8a78-6b1c29c6e2a0 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:53.625393 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9897d072-076c-48c2-8a78-6b1c29c6e2a0 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:53.626673 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 665/1354] 10.222.0.33 () {54 vars in 921 bytes} [Mon Oct 3 12:41:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:41:53.631906 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d5cf8-1c94-460c-870f-ffd236057307 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:53.633295 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d5cf8-1c94-460c-870f-ffd236057307 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:53.633295 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d5cf8-1c94-460c-870f-ffd236057307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:53.710709 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-073d5cf8-1c94-460c-870f-ffd236057307 None None] MFA Rules not processed for user `25eef64e92e9422ea62508c3eb0a197e`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:41:53.711925 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-073d5cf8-1c94-460c-870f-ffd236057307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:53.753980 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:41:53.819416 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 690/1355] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Oct 3 12:41:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:41:57.622601 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9c648-01d7-4a7d-b201-0a71baa63caf None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:57.623132 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9c648-01d7-4a7d-b201-0a71baa63caf None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:57.623132 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9c648-01d7-4a7d-b201-0a71baa63caf None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:57.631025 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 666/1356] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:41:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:41:57.631244 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48bf228-e16c-403c-9144-a6422a288086 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:57.632020 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48bf228-e16c-403c-9144-a6422a288086 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:57.632145 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48bf228-e16c-403c-9144-a6422a288086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:57.710826 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-d48bf228-e16c-403c-9144-a6422a288086 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:41:57.712169 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-d48bf228-e16c-403c-9144-a6422a288086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:57.823907 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 691/1357] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:41:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:41:58.129109 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa94885-77dd-4805-b1e1-54416e0da2ac None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:58.129885 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa94885-77dd-4805-b1e1-54416e0da2ac None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:58.130395 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa94885-77dd-4805-b1e1-54416e0da2ac None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:58.132033 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 667/1358] 10.222.0.33 () {54 vars in 961 bytes} [Mon Oct 3 12:41:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:41:58.144960 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ee0a8-40ef-4d53-8af0-0a487c0bd629 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:58.145982 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ee0a8-40ef-4d53-8af0-0a487c0bd629 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:58.146502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ee0a8-40ef-4d53-8af0-0a487c0bd629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:58.166479 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:41:58.366784 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-4b7ee0a8-40ef-4d53-8af0-0a487c0bd629 None None] MFA Rules not processed for user `990f5837994f4a088c656c9b33e9a0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:41:58.368061 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-4b7ee0a8-40ef-4d53-8af0-0a487c0bd629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:58.500211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 692/1359] 10.222.0.33 () {58 vars in 1058 bytes} [Mon Oct 3 12:41:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:41:58.508717 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa94885-77dd-4805-b1e1-54416e0da2ac None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:41:58.510311 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-bfa94885-77dd-4805-b1e1-54416e0da2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:58.664950 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa94885-77dd-4805-b1e1-54416e0da2ac None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:41:58.742732 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:41:58.789088 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5760df-4b45-4add-9790-5fdaffc06493 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:41:58.789761 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5760df-4b45-4add-9790-5fdaffc06493 service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:41:58.790046 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5760df-4b45-4add-9790-5fdaffc06493 service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:41:58.790306 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5760df-4b45-4add-9790-5fdaffc06493 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:41:58.792057 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ae5760df-4b45-4add-9790-5fdaffc06493 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:41:58.891995 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5760df-4b45-4add-9790-5fdaffc06493 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:41:58.893637 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5760df-4b45-4add-9790-5fdaffc06493 service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:41:58.913078 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 668/1360] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:41:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:42:01.363343 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a370e0-ec5a-4e88-a3b5-cbebd3d2209c None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:01.364242 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a370e0-ec5a-4e88-a3b5-cbebd3d2209c None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:01.364596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a370e0-ec5a-4e88-a3b5-cbebd3d2209c None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:01.365928 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 693/1361] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:42:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:42:01.371687 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f65e14-71c3-4056-8943-966c9426db29 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:01.372129 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f65e14-71c3-4056-8943-966c9426db29 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:01.372236 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f65e14-71c3-4056-8943-966c9426db29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:01.446493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-93f65e14-71c3-4056-8943-966c9426db29 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:42:01.447946 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-93f65e14-71c3-4056-8943-966c9426db29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:01.491895 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:01.555059 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 669/1362] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:42:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:42:01.865800 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a370e0-ec5a-4e88-a3b5-cbebd3d2209c None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:42:01.867349 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-97a370e0-ec5a-4e88-a3b5-cbebd3d2209c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:01.952586 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:02.043900 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a370e0-ec5a-4e88-a3b5-cbebd3d2209c None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:42:02.163536 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b42171-c5d7-454d-ab99-ac1b0e0e7ace service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:42:02.164291 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b42171-c5d7-454d-ab99-ac1b0e0e7ace service nova] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:02.164640 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b42171-c5d7-454d-ab99-ac1b0e0e7ace service nova] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:02.164979 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b42171-c5d7-454d-ab99-ac1b0e0e7ace service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:02.166646 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-09b42171-c5d7-454d-ab99-ac1b0e0e7ace service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:02.190759 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:02.266594 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b42171-c5d7-454d-ab99-ac1b0e0e7ace service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:42:02.268893 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b42171-c5d7-454d-ab99-ac1b0e0e7ace service nova] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:42:02.289091 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 694/1363] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:42:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:42:04.959008 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f28afd-7566-4aa9-90cc-0eba6d43ed4a None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:04.959585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f28afd-7566-4aa9-90cc-0eba6d43ed4a None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:04.959585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f28afd-7566-4aa9-90cc-0eba6d43ed4a None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:04.960919 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 670/1364] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:42:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:42:04.966443 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42a25e-6c51-4326-85ab-a4523dd00896 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:04.966874 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42a25e-6c51-4326-85ab-a4523dd00896 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:04.967025 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42a25e-6c51-4326-85ab-a4523dd00896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:05.038536 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-be42a25e-6c51-4326-85ab-a4523dd00896 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:42:05.039771 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-be42a25e-6c51-4326-85ab-a4523dd00896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:05.145164 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:05.147839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 695/1365] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:42:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:42:05.452342 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d668e-d529-4711-a31b-80c2b5f3ea37 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:05.452814 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d668e-d529-4711-a31b-80c2b5f3ea37 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:05.453095 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d668e-d529-4711-a31b-80c2b5f3ea37 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:05.454609 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 671/1366] 10.222.0.33 () {54 vars in 961 bytes} [Mon Oct 3 12:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:42:05.461755 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b66f5-10f2-456f-b524-70502f66046b None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:05.462095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b66f5-10f2-456f-b524-70502f66046b None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:05.462383 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b66f5-10f2-456f-b524-70502f66046b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:05.543443 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-095b66f5-10f2-456f-b524-70502f66046b None None] MFA Rules not processed for user `990f5837994f4a088c656c9b33e9a0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:42:05.544535 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-095b66f5-10f2-456f-b524-70502f66046b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:05.668284 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 696/1367] 10.222.0.33 () {58 vars in 1058 bytes} [Mon Oct 3 12:42:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:42:05.676698 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d668e-d529-4711-a31b-80c2b5f3ea37 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:42:05.678194 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a88d668e-d529-4711-a31b-80c2b5f3ea37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:05.837133 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d668e-d529-4711-a31b-80c2b5f3ea37 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:42:05.860477 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:05.966429 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b61d7d3-e5fa-4e29-acd4-cd7ce265623a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:42:05.967130 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61d7d3-e5fa-4e29-acd4-cd7ce265623a service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:05.967455 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61d7d3-e5fa-4e29-acd4-cd7ce265623a service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:05.967625 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61d7d3-e5fa-4e29-acd4-cd7ce265623a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:05.969154 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1b61d7d3-e5fa-4e29-acd4-cd7ce265623a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:05.999672 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156528f2-f6d3-4811-80df-456fea6867ff None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:06.000194 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156528f2-f6d3-4811-80df-456fea6867ff None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:06.000306 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156528f2-f6d3-4811-80df-456fea6867ff None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:06.001761 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 697/1368] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 12:42:06.007464 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968ff38-b2ab-4b8b-8dc5-87b1d13812f2 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:06.007752 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968ff38-b2ab-4b8b-8dc5-87b1d13812f2 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:06.008030 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968ff38-b2ab-4b8b-8dc5-87b1d13812f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:06.077038 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:06.081139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b61d7d3-e5fa-4e29-acd4-cd7ce265623a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:42:06.082833 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b61d7d3-e5fa-4e29-acd4-cd7ce265623a service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:42:06.086939 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-1968ff38-b2ab-4b8b-8dc5-87b1d13812f2 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:42:06.091374 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-1968ff38-b2ab-4b8b-8dc5-87b1d13812f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:06.104502 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 672/1369] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:42:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:42:06.191572 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 698/1370] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:42:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:42:06.531087 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e2c6f-5851-4935-a184-cb56d6840ddb None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:06.531087 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e2c6f-5851-4935-a184-cb56d6840ddb None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:06.531087 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e2c6f-5851-4935-a184-cb56d6840ddb None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:06.532673 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 673/1371] 10.222.0.33 () {54 vars in 963 bytes} [Mon Oct 3 12:42:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:42:06.537970 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9774a6-fb3d-4f00-a06e-b41d0827b45c None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:06.538406 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9774a6-fb3d-4f00-a06e-b41d0827b45c None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:06.538783 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9774a6-fb3d-4f00-a06e-b41d0827b45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:06.626371 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-1b9774a6-fb3d-4f00-a06e-b41d0827b45c None None] MFA Rules not processed for user `524e9184eee1486cb071cad5c67050c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:42:06.628403 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-1b9774a6-fb3d-4f00-a06e-b41d0827b45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:06.702490 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:06.710877 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 699/1372] 10.222.0.33 () {58 vars in 1060 bytes} [Mon Oct 3 12:42:06 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:42:06.718617 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e2c6f-5851-4935-a184-cb56d6840ddb None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:42:06.720377 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ee3e2c6f-5851-4935-a184-cb56d6840ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:06.753591 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:06.836080 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e2c6f-5851-4935-a184-cb56d6840ddb None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:42:06.923173 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8009c9f-8eee-4063-882f-e85f482a80e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:42:06.924937 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8009c9f-8eee-4063-882f-e85f482a80e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:06.925222 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8009c9f-8eee-4063-882f-e85f482a80e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:06.925497 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8009c9f-8eee-4063-882f-e85f482a80e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:06.927111 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a8009c9f-8eee-4063-882f-e85f482a80e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:07.010891 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:07.056339 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8009c9f-8eee-4063-882f-e85f482a80e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:42:07.058196 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8009c9f-8eee-4063-882f-e85f482a80e8 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:42:07.085304 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 674/1373] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:42:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:42:08.503635 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4a0b1-3233-47a8-abe9-31abff940ef1 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:08.504227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4a0b1-3233-47a8-abe9-31abff940ef1 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:08.504227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4a0b1-3233-47a8-abe9-31abff940ef1 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:08.505547 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 700/1374] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:42:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:42:08.511197 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85549b48-f256-40e5-bf39-b2b050b1d32d None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:08.511652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85549b48-f256-40e5-bf39-b2b050b1d32d None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:08.514237 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85549b48-f256-40e5-bf39-b2b050b1d32d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:08.579571 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-85549b48-f256-40e5-bf39-b2b050b1d32d None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:42:08.581699 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-85549b48-f256-40e5-bf39-b2b050b1d32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:08.680944 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 675/1375] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:42:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:42:08.975351 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4a0b1-3233-47a8-abe9-31abff940ef1 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:42:08.976973 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-a1b4a0b1-3233-47a8-abe9-31abff940ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:09.130946 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4a0b1-3233-47a8-abe9-31abff940ef1 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:42:09.200226 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:09.255272 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8786112d-3e48-411a-aaca-32251c226039 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:42:09.255272 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786112d-3e48-411a-aaca-32251c226039 service nova] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:09.255687 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786112d-3e48-411a-aaca-32251c226039 service nova] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:09.255808 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786112d-3e48-411a-aaca-32251c226039 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:09.257350 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-8786112d-3e48-411a-aaca-32251c226039 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:09.359163 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8786112d-3e48-411a-aaca-32251c226039 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:42:09.359163 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8786112d-3e48-411a-aaca-32251c226039 service nova] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:42:09.382907 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 701/1376] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:42:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:42:11.780236 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707c36b-e5e1-4ff7-a4ac-d28fae4e3217 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:11.780759 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707c36b-e5e1-4ff7-a4ac-d28fae4e3217 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:11.780843 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707c36b-e5e1-4ff7-a4ac-d28fae4e3217 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:11.782278 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 676/1377] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:42:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:42:11.788051 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c82eb-e712-4a92-99e2-15fd8e84b408 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:11.788463 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c82eb-e712-4a92-99e2-15fd8e84b408 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:11.788662 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c82eb-e712-4a92-99e2-15fd8e84b408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:11.858252 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-7f4c82eb-e712-4a92-99e2-15fd8e84b408 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:42:11.859505 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-7f4c82eb-e712-4a92-99e2-15fd8e84b408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:11.891765 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:11.963449 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 702/1378] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:42:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:42:12.265253 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5707c36b-e5e1-4ff7-a4ac-d28fae4e3217 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:42:12.266763 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-5707c36b-e5e1-4ff7-a4ac-d28fae4e3217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:12.309860 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:12.426354 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5707c36b-e5e1-4ff7-a4ac-d28fae4e3217 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:42:12.545081 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:12.550689 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb07dc-7947-41d2-a857-2750facc31a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:42:12.551416 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb07dc-7947-41d2-a857-2750facc31a2 service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:12.551757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb07dc-7947-41d2-a857-2750facc31a2 service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:12.552097 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb07dc-7947-41d2-a857-2750facc31a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:12.553831 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-9adb07dc-7947-41d2-a857-2750facc31a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:12.650992 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adb07dc-7947-41d2-a857-2750facc31a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:42:12.652736 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adb07dc-7947-41d2-a857-2750facc31a2 service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:42:12.671799 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 677/1379] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:42:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:42:15.127367 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ccbe1c-3c4a-49f3-aaf1-646d1d2f505f None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:15.127901 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ccbe1c-3c4a-49f3-aaf1-646d1d2f505f None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:15.128156 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ccbe1c-3c4a-49f3-aaf1-646d1d2f505f None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:15.130092 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 703/1380] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:42:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:42:15.136749 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a2208-176a-4e6d-a775-86292820203f None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:15.137218 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a2208-176a-4e6d-a775-86292820203f None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:15.137446 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a2208-176a-4e6d-a775-86292820203f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:15.205508 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-6c5a2208-176a-4e6d-a775-86292820203f None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:42:15.206745 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-6c5a2208-176a-4e6d-a775-86292820203f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:15.286234 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:15.320890 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 678/1381] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:42:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:42:15.697371 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ccbe1c-3c4a-49f3-aaf1-646d1d2f505f None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:42:15.698991 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-99ccbe1c-3c4a-49f3-aaf1-646d1d2f505f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:15.852373 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ccbe1c-3c4a-49f3-aaf1-646d1d2f505f None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:42:15.866275 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:15.973555 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a27c2-9b30-4664-91ad-2a6c982fb9b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:42:15.974340 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a27c2-9b30-4664-91ad-2a6c982fb9b8 service nova] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:15.974625 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a27c2-9b30-4664-91ad-2a6c982fb9b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:15.974895 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a27c2-9b30-4664-91ad-2a6c982fb9b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:15.976517 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-9a5a27c2-9b30-4664-91ad-2a6c982fb9b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:16.070301 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5a27c2-9b30-4664-91ad-2a6c982fb9b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:42:16.071993 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5a27c2-9b30-4664-91ad-2a6c982fb9b8 service nova] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:42:16.091025 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 704/1382] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:42:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:42:18.534134 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c29bc2-f7f5-4c3d-b99a-9527068134b9 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:18.534644 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c29bc2-f7f5-4c3d-b99a-9527068134b9 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:18.534644 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c29bc2-f7f5-4c3d-b99a-9527068134b9 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:18.535980 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 679/1383] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:42:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:42:18.541516 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49db9d2-2c33-47cb-8908-1d941bea6d81 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:18.541991 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49db9d2-2c33-47cb-8908-1d941bea6d81 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:18.542108 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49db9d2-2c33-47cb-8908-1d941bea6d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:18.565000 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:18.613341 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-a49db9d2-2c33-47cb-8908-1d941bea6d81 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:42:18.614570 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-a49db9d2-2c33-47cb-8908-1d941bea6d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:18.716400 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 705/1384] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:42:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:42:19.039886 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c29bc2-f7f5-4c3d-b99a-9527068134b9 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:42:19.042209 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-87c29bc2-f7f5-4c3d-b99a-9527068134b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:19.378689 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c29bc2-f7f5-4c3d-b99a-9527068134b9 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:42:19.431161 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:19.495973 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1ac279-27c9-48ba-8ad7-e0685888c8b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:42:19.496591 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ac279-27c9-48ba-8ad7-e0685888c8b7 service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:19.496870 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ac279-27c9-48ba-8ad7-e0685888c8b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:19.497159 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ac279-27c9-48ba-8ad7-e0685888c8b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:19.498751 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-bb1ac279-27c9-48ba-8ad7-e0685888c8b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:19.588309 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1ac279-27c9-48ba-8ad7-e0685888c8b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:42:19.589819 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1ac279-27c9-48ba-8ad7-e0685888c8b7 service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:42:19.607767 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 680/1385] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:42:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:42:21.746120 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80b1ba-412a-4d67-9e49-95205b9e9ae5 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:21.746645 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80b1ba-412a-4d67-9e49-95205b9e9ae5 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:21.746741 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80b1ba-412a-4d67-9e49-95205b9e9ae5 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:21.748579 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 706/1386] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:42:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:42:21.754326 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b2cc8-f793-4a77-8b82-0d250745b938 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:21.754918 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b2cc8-f793-4a77-8b82-0d250745b938 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:21.754918 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b2cc8-f793-4a77-8b82-0d250745b938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:21.823727 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-0d5b2cc8-f793-4a77-8b82-0d250745b938 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:42:21.825012 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-0d5b2cc8-f793-4a77-8b82-0d250745b938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:21.834817 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:21.931513 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 681/1387] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:42:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:42:22.240748 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d80b1ba-412a-4d67-9e49-95205b9e9ae5 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:42:22.243174 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-4d80b1ba-412a-4d67-9e49-95205b9e9ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:22.343092 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:22.406416 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d80b1ba-412a-4d67-9e49-95205b9e9ae5 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:42:22.528010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0c3ad5-766b-4de2-812b-b74c2d6a0759 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:42:22.528647 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c3ad5-766b-4de2-812b-b74c2d6a0759 service nova] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:22.528931 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c3ad5-766b-4de2-812b-b74c2d6a0759 service nova] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:22.529214 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c3ad5-766b-4de2-812b-b74c2d6a0759 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:22.530875 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-3e0c3ad5-766b-4de2-812b-b74c2d6a0759 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:22.594866 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:22.627913 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0c3ad5-766b-4de2-812b-b74c2d6a0759 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:42:22.629590 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0c3ad5-766b-4de2-812b-b74c2d6a0759 service nova] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:42:22.650051 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 707/1388] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:42:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:42:25.057988 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc06553-c5fc-4cbd-9a85-56426b60f803 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:25.058558 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc06553-c5fc-4cbd-9a85-56426b60f803 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:25.058558 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc06553-c5fc-4cbd-9a85-56426b60f803 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:25.059911 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 682/1389] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:42:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:42:25.066151 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62659f59-a1f6-410f-a20d-bf5afddafa97 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:25.066547 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62659f59-a1f6-410f-a20d-bf5afddafa97 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:25.066774 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62659f59-a1f6-410f-a20d-bf5afddafa97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:25.133719 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-62659f59-a1f6-410f-a20d-bf5afddafa97 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:42:25.134914 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-62659f59-a1f6-410f-a20d-bf5afddafa97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:25.240778 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 708/1390] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:42:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:42:25.553482 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc06553-c5fc-4cbd-9a85-56426b60f803 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:42:25.554963 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-bcc06553-c5fc-4cbd-9a85-56426b60f803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:25.707428 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:25.713841 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc06553-c5fc-4cbd-9a85-56426b60f803 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:42:25.778175 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d06d11-7b7d-4e4c-8bbd-03c800b3d0bd None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:25.778520 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d06d11-7b7d-4e4c-8bbd-03c800b3d0bd None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:25.778796 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d06d11-7b7d-4e4c-8bbd-03c800b3d0bd None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:25.780164 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 709/1391] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:42:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 12:42:25.785101 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459bea8-5e50-48eb-9e2a-b35f21b1d41a None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:25.785382 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459bea8-5e50-48eb-9e2a-b35f21b1d41a None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:25.785650 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459bea8-5e50-48eb-9e2a-b35f21b1d41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:25.830343 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:25.832051 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d243d139-5f0c-45bb-8025-2a1c76be7dbe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:42:25.832622 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243d139-5f0c-45bb-8025-2a1c76be7dbe service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:25.832897 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243d139-5f0c-45bb-8025-2a1c76be7dbe service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:25.833164 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243d139-5f0c-45bb-8025-2a1c76be7dbe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:25.834802 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d243d139-5f0c-45bb-8025-2a1c76be7dbe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:25.855263 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-3459bea8-5e50-48eb-9e2a-b35f21b1d41a None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:42:25.857157 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-3459bea8-5e50-48eb-9e2a-b35f21b1d41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:25.927163 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d243d139-5f0c-45bb-8025-2a1c76be7dbe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:42:25.928782 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d243d139-5f0c-45bb-8025-2a1c76be7dbe service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:42:25.942973 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:25.947266 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 683/1392] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:42:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:42:25.961030 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 710/1393] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:42:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:42:26.280627 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ad175-31cc-41a5-a78e-70e92aa99b84 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:26.281164 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ad175-31cc-41a5-a78e-70e92aa99b84 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:26.281274 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ad175-31cc-41a5-a78e-70e92aa99b84 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:26.282754 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 684/1394] 10.222.0.33 () {54 vars in 962 bytes} [Mon Oct 3 12:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:42:26.288298 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3940871-e515-4307-82f9-df0d0eb4abe4 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:26.288797 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3940871-e515-4307-82f9-df0d0eb4abe4 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:26.288892 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3940871-e515-4307-82f9-df0d0eb4abe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:26.366624 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-d3940871-e515-4307-82f9-df0d0eb4abe4 None None] MFA Rules not processed for user `b697e4847b13496d98a543675f5df1c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:42:26.367762 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-d3940871-e515-4307-82f9-df0d0eb4abe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:26.430312 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:26.451102 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 711/1395] 10.222.0.33 () {58 vars in 1059 bytes} [Mon Oct 3 12:42:26 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:42:26.460276 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ad175-31cc-41a5-a78e-70e92aa99b84 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:42:26.461860 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-601ad175-31cc-41a5-a78e-70e92aa99b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:26.565939 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ad175-31cc-41a5-a78e-70e92aa99b84 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:42:26.643124 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d595c88-0334-4062-9aa2-49f82352166f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b697e4847b13496d98a543675f5df1c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:42:26.643729 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d595c88-0334-4062-9aa2-49f82352166f service glance] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:26.644002 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d595c88-0334-4062-9aa2-49f82352166f service glance] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:26.644270 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d595c88-0334-4062-9aa2-49f82352166f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:26.645812 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-8d595c88-0334-4062-9aa2-49f82352166f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:26.697218 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:26.758516 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d595c88-0334-4062-9aa2-49f82352166f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:42:26.760247 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d595c88-0334-4062-9aa2-49f82352166f service glance] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:42:26.779570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 685/1396] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:42:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:42:27.700789 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca18f201-fd60-4a61-8dad-0b89ea75848d None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:27.701316 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca18f201-fd60-4a61-8dad-0b89ea75848d None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:27.701423 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca18f201-fd60-4a61-8dad-0b89ea75848d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:27.770022 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-ca18f201-fd60-4a61-8dad-0b89ea75848d None None] MFA Rules not processed for user `2e6e4387a7c24bbc82c22eefe2a3d138`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:42:27.771256 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-ca18f201-fd60-4a61-8dad-0b89ea75848d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:27.861900 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 712/1397] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Oct 3 12:42:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:42:27.903425 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d559a9a-776e-45b0-9d75-2d928ccefc66 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:27.903929 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d559a9a-776e-45b0-9d75-2d928ccefc66 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:27.904042 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d559a9a-776e-45b0-9d75-2d928ccefc66 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:27.905391 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 686/1398] 10.222.0.33 () {54 vars in 943 bytes} [Mon Oct 3 12:42:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:42:27.912794 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a46803-91c9-4966-95f5-0aaf46ccd4e8 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:27.913084 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a46803-91c9-4966-95f5-0aaf46ccd4e8 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:27.913357 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a46803-91c9-4966-95f5-0aaf46ccd4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:27.989400 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-d8a46803-91c9-4966-95f5-0aaf46ccd4e8 None None] MFA Rules not processed for user `b111d72a34144073917de4f566a58233`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:42:27.990635 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-d8a46803-91c9-4966-95f5-0aaf46ccd4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:28.005530 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:28.098200 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 713/1399] 10.222.0.33 () {58 vars in 1040 bytes} [Mon Oct 3 12:42:27 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:42:28.107580 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d559a9a-776e-45b0-9d75-2d928ccefc66 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:42:28.109060 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-5d559a9a-776e-45b0-9d75-2d928ccefc66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:28.266179 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d559a9a-776e-45b0-9d75-2d928ccefc66 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:42:28.291673 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:28.315397 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161e6b3-cc04-4bcf-a75f-c6ee506ccc40 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:28.315691 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161e6b3-cc04-4bcf-a75f-c6ee506ccc40 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:28.315971 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161e6b3-cc04-4bcf-a75f-c6ee506ccc40 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:28.317297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 714/1400] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:42:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 12:42:28.322681 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68e004-b5d6-4504-8c9b-6d38b7c233d6 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:28.322960 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68e004-b5d6-4504-8c9b-6d38b7c233d6 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:28.323229 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68e004-b5d6-4504-8c9b-6d38b7c233d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:28.380060 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd7d775-d71c-4109-9954-3b2fb0da76d1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b111d72a34144073917de4f566a58233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:42:28.380673 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7d775-d71c-4109-9954-3b2fb0da76d1 service swift] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:28.380991 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7d775-d71c-4109-9954-3b2fb0da76d1 service swift] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:28.381337 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7d775-d71c-4109-9954-3b2fb0da76d1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:28.382885 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-9dd7d775-d71c-4109-9954-3b2fb0da76d1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:28.395630 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-5f68e004-b5d6-4504-8c9b-6d38b7c233d6 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:42:28.396832 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-5f68e004-b5d6-4504-8c9b-6d38b7c233d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:28.470993 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd7d775-d71c-4109-9954-3b2fb0da76d1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:42:28.472597 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd7d775-d71c-4109-9954-3b2fb0da76d1 service swift] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:42:28.478794 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:28.481141 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 687/1401] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Oct 3 12:42:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 12:42:28.501114 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 715/1402] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:42:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:42:28.835802 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd7d775-d71c-4109-9954-3b2fb0da76d1 service swift] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:42:28.837000 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-9dd7d775-d71c-4109-9954-3b2fb0da76d1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:28.901601 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:28.940818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd7d775-d71c-4109-9954-3b2fb0da76d1 service swift] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:42:29.080416 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c185d-0d07-4e43-af34-11b2171a53f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:42:29.081033 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c185d-0d07-4e43-af34-11b2171a53f9 service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:29.081307 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c185d-0d07-4e43-af34-11b2171a53f9 service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:29.081574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c185d-0d07-4e43-af34-11b2171a53f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:29.083158 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-5b3c185d-0d07-4e43-af34-11b2171a53f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:29.165713 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:29.190261 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3c185d-0d07-4e43-af34-11b2171a53f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:42:29.191897 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3c185d-0d07-4e43-af34-11b2171a53f9 service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:42:29.212917 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 688/1403] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:42:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:42:32.667921 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba187b29-e20e-4b75-bfb0-b1a7cbd43ad4 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:32.668542 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba187b29-e20e-4b75-bfb0-b1a7cbd43ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:32.668542 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba187b29-e20e-4b75-bfb0-b1a7cbd43ad4 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:32.669809 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 716/1404] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:42:32.675130 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faecc31-0bdf-4e59-8b4f-34ce3764d35a None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:32.675597 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faecc31-0bdf-4e59-8b4f-34ce3764d35a None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:32.675707 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faecc31-0bdf-4e59-8b4f-34ce3764d35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:32.747585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-5faecc31-0bdf-4e59-8b4f-34ce3764d35a None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:42:32.748913 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-5faecc31-0bdf-4e59-8b4f-34ce3764d35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:32.855813 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 689/1405] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:42:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:42:33.147822 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba187b29-e20e-4b75-bfb0-b1a7cbd43ad4 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:42:33.149368 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-ba187b29-e20e-4b75-bfb0-b1a7cbd43ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:33.330992 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba187b29-e20e-4b75-bfb0-b1a7cbd43ad4 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:42:33.408942 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:33.484576 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2141a64f-2b8e-499e-b97f-1caac8d06748 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:42:33.485549 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141a64f-2b8e-499e-b97f-1caac8d06748 service nova] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:33.486093 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141a64f-2b8e-499e-b97f-1caac8d06748 service nova] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:33.486498 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141a64f-2b8e-499e-b97f-1caac8d06748 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:33.488517 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-2141a64f-2b8e-499e-b97f-1caac8d06748 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:33.628511 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2141a64f-2b8e-499e-b97f-1caac8d06748 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:42:33.630216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2141a64f-2b8e-499e-b97f-1caac8d06748 service nova] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:42:33.663105 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 717/1406] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:42:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:42:36.274288 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e2d9e-5b1f-42b9-9b3c-c0bc587c4bef None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:36.274915 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e2d9e-5b1f-42b9-9b3c-c0bc587c4bef None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:36.274915 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e2d9e-5b1f-42b9-9b3c-c0bc587c4bef None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:36.276609 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 690/1407] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:42:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:42:36.281822 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638076a6-137f-4378-9bf3-32da882002b9 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:36.282264 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638076a6-137f-4378-9bf3-32da882002b9 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:36.282925 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638076a6-137f-4378-9bf3-32da882002b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:36.356619 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-638076a6-137f-4378-9bf3-32da882002b9 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:42:36.357899 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-638076a6-137f-4378-9bf3-32da882002b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:36.386038 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:36.465699 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 718/1408] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:42:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:42:36.757756 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e2d9e-5b1f-42b9-9b3c-c0bc587c4bef None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:42:36.759274 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-227e2d9e-5b1f-42b9-9b3c-c0bc587c4bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:36.813392 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:36.916518 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e2d9e-5b1f-42b9-9b3c-c0bc587c4bef None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:42:37.042705 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f501237f-18a5-4b7a-9d68-e39bc5233a7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:42:37.042705 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501237f-18a5-4b7a-9d68-e39bc5233a7c service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:37.042705 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501237f-18a5-4b7a-9d68-e39bc5233a7c service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:37.042705 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501237f-18a5-4b7a-9d68-e39bc5233a7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:37.043260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-f501237f-18a5-4b7a-9d68-e39bc5233a7c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:37.052432 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:37.142643 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f501237f-18a5-4b7a-9d68-e39bc5233a7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:42:37.144367 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f501237f-18a5-4b7a-9d68-e39bc5233a7c service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:42:37.164288 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 691/1409] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:42:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:42:46.464298 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=34 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:47.161576 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=41 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:52.462272 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57498ef9-db2b-4cf5-a386-7aa0fbe44b54 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:52.462804 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57498ef9-db2b-4cf5-a386-7aa0fbe44b54 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:52.462804 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57498ef9-db2b-4cf5-a386-7aa0fbe44b54 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:52.464154 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 719/1410] 10.222.0.33 () {54 vars in 918 bytes} [Mon Oct 3 12:42:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:42:52.469093 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9fd40-7c65-443b-9afa-fd9f79f4b982 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:52.469541 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9fd40-7c65-443b-9afa-fd9f79f4b982 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:52.469873 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9fd40-7c65-443b-9afa-fd9f79f4b982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:52.545245 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-e1c9fd40-7c65-443b-9afa-fd9f79f4b982 None None] MFA Rules not processed for user `25eef64e92e9422ea62508c3eb0a197e`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:42:52.546440 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e1c9fd40-7c65-443b-9afa-fd9f79f4b982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:52.652891 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 692/1411] 10.222.0.33 () {58 vars in 1015 bytes} [Mon Oct 3 12:42:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:42:52.667215 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7272072-ff1d-401d-ad4d-a2816ce2d56a None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:52.667544 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7272072-ff1d-401d-ad4d-a2816ce2d56a None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:52.667817 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7272072-ff1d-401d-ad4d-a2816ce2d56a None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:52.669155 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 720/1412] 10.222.0.33 () {54 vars in 918 bytes} [Mon Oct 3 12:42:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:42:52.674163 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c844115-4c62-4bd6-b73c-d80aa70cb7d0 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:52.674437 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c844115-4c62-4bd6-b73c-d80aa70cb7d0 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:52.674705 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c844115-4c62-4bd6-b73c-d80aa70cb7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:52.749407 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:52.751928 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-9c844115-4c62-4bd6-b73c-d80aa70cb7d0 None None] MFA Rules not processed for user `990f5837994f4a088c656c9b33e9a0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:42:52.753252 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-9c844115-4c62-4bd6-b73c-d80aa70cb7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:52.870416 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 693/1413] 10.222.0.33 () {58 vars in 1015 bytes} [Mon Oct 3 12:42:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:42:55.512910 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04468aa-3f93-4340-8eb7-3175b7d78269 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:55.513454 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04468aa-3f93-4340-8eb7-3175b7d78269 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:55.513454 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04468aa-3f93-4340-8eb7-3175b7d78269 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:55.514790 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 721/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 12:42:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:42:55.520798 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cba4ca-f102-4610-9f7b-51b65ef9214a None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:55.521246 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cba4ca-f102-4610-9f7b-51b65ef9214a None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:55.521355 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cba4ca-f102-4610-9f7b-51b65ef9214a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:55.586020 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-90cba4ca-f102-4610-9f7b-51b65ef9214a None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:42:55.587205 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-90cba4ca-f102-4610-9f7b-51b65ef9214a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:55.641055 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:55.682966 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 694/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 12:42:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:42:55.987371 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04468aa-3f93-4340-8eb7-3175b7d78269 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:42:55.988905 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-a04468aa-3f93-4340-8eb7-3175b7d78269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:56.132917 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04468aa-3f93-4340-8eb7-3175b7d78269 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:42:56.204654 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:56.245149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1712633-6d66-4378-ae63-0d69953671b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:42:56.246160 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1712633-6d66-4378-ae63-0d69953671b4 service nova] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:56.247004 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1712633-6d66-4378-ae63-0d69953671b4 service nova] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:56.247277 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1712633-6d66-4378-ae63-0d69953671b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:56.248981 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-e1712633-6d66-4378-ae63-0d69953671b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:56.339283 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1712633-6d66-4378-ae63-0d69953671b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:42:56.340888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1712633-6d66-4378-ae63-0d69953671b4 service nova] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:42:56.358663 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 722/1416] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:42:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:42:57.521657 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb374ca2-5960-4fb3-b1f3-5740a41d4d35 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:57.522199 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb374ca2-5960-4fb3-b1f3-5740a41d4d35 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:57.522199 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb374ca2-5960-4fb3-b1f3-5740a41d4d35 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:57.523452 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 695/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 12:42:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:42:57.529674 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e2b695-1f35-480d-9418-194b976e6179 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:57.530133 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e2b695-1f35-480d-9418-194b976e6179 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:57.530248 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e2b695-1f35-480d-9418-194b976e6179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:57.603504 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-27e2b695-1f35-480d-9418-194b976e6179 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:42:57.604701 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-27e2b695-1f35-480d-9418-194b976e6179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:57.631184 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:57.698245 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 723/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 12:42:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:42:57.848733 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb374ca2-5960-4fb3-b1f3-5740a41d4d35 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:42:57.850489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-fb374ca2-5960-4fb3-b1f3-5740a41d4d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:57.998006 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb374ca2-5960-4fb3-b1f3-5740a41d4d35 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:42:58.037836 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:58.110092 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf3ca53-07ad-4ad3-82f9-84a294a04517 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:42:58.111343 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf3ca53-07ad-4ad3-82f9-84a294a04517 service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:58.111622 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf3ca53-07ad-4ad3-82f9-84a294a04517 service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:58.111893 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf3ca53-07ad-4ad3-82f9-84a294a04517 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:58.113437 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-8bf3ca53-07ad-4ad3-82f9-84a294a04517 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:58.200399 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf3ca53-07ad-4ad3-82f9-84a294a04517 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:42:58.201979 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf3ca53-07ad-4ad3-82f9-84a294a04517 service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:42:58.220356 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 696/1419] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:42:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:42:59.356145 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbf1d2-f340-4715-91c4-b7cc72d27542 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:59.356735 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbf1d2-f340-4715-91c4-b7cc72d27542 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:59.357065 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbf1d2-f340-4715-91c4-b7cc72d27542 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:59.359525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 724/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 12:42:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:42:59.366456 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45ede8-2183-4b49-864c-c9221db7fbc9 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:59.366960 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45ede8-2183-4b49-864c-c9221db7fbc9 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:59.367087 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45ede8-2183-4b49-864c-c9221db7fbc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:59.422010 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:59.443342 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-1a45ede8-2183-4b49-864c-c9221db7fbc9 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:42:59.445166 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1a45ede8-2183-4b49-864c-c9221db7fbc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:59.539635 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 697/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 12:42:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:42:59.692482 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbf1d2-f340-4715-91c4-b7cc72d27542 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:42:59.693996 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-51dbf1d2-f340-4715-91c4-b7cc72d27542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:42:59.841811 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbf1d2-f340-4715-91c4-b7cc72d27542 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:42:59.854636 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:42:59.957503 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff32bfb-1b11-4271-a3e0-9b8d98d0728b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:42:59.958109 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff32bfb-1b11-4271-a3e0-9b8d98d0728b service nova] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:42:59.958387 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff32bfb-1b11-4271-a3e0-9b8d98d0728b service nova] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:42:59.958657 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff32bfb-1b11-4271-a3e0-9b8d98d0728b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:42:59.960267 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-bff32bfb-1b11-4271-a3e0-9b8d98d0728b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:00.047985 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff32bfb-1b11-4271-a3e0-9b8d98d0728b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:43:00.049675 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff32bfb-1b11-4271-a3e0-9b8d98d0728b service nova] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:43:00.068144 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 725/1422] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:42:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:43:01.171704 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae8a310-4520-416a-8faf-621ff6214f44 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:01.172283 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae8a310-4520-416a-8faf-621ff6214f44 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:01.172283 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae8a310-4520-416a-8faf-621ff6214f44 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:01.173675 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 698/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 12:43:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:43:01.179873 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d7436-e516-43f9-8c14-e3f32bfc9d35 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:01.180320 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d7436-e516-43f9-8c14-e3f32bfc9d35 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:01.180515 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d7436-e516-43f9-8c14-e3f32bfc9d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:01.201150 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:43:01.262106 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-418d7436-e516-43f9-8c14-e3f32bfc9d35 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:43:01.263514 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-418d7436-e516-43f9-8c14-e3f32bfc9d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:01.361828 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 726/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 12:43:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:43:01.515019 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae8a310-4520-416a-8faf-621ff6214f44 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:43:01.517174 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-5ae8a310-4520-416a-8faf-621ff6214f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:01.625732 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:43:01.661824 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae8a310-4520-416a-8faf-621ff6214f44 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:43:01.773155 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a8174e-f040-4e77-937a-c76b5f5c3b6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:43:01.773674 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a8174e-f040-4e77-937a-c76b5f5c3b6a service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:01.773946 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a8174e-f040-4e77-937a-c76b5f5c3b6a service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:01.774211 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a8174e-f040-4e77-937a-c76b5f5c3b6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:01.775868 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-96a8174e-f040-4e77-937a-c76b5f5c3b6a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:01.850969 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:43:01.864245 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a8174e-f040-4e77-937a-c76b5f5c3b6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:43:01.865925 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a8174e-f040-4e77-937a-c76b5f5c3b6a service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:43:01.884541 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 699/1425] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:43:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:43:02.996746 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1d51d-b18b-465e-8a78-8aaabea8b7c5 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:02.997319 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1d51d-b18b-465e-8a78-8aaabea8b7c5 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:02.997319 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1d51d-b18b-465e-8a78-8aaabea8b7c5 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:02.998649 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 727/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 12:43:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:43:03.005583 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c404a-0807-4ac4-b208-fcb57550dd13 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:03.006015 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c404a-0807-4ac4-b208-fcb57550dd13 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:03.006153 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c404a-0807-4ac4-b208-fcb57550dd13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:03.082948 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-5d9c404a-0807-4ac4-b208-fcb57550dd13 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:43:03.084164 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-5d9c404a-0807-4ac4-b208-fcb57550dd13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:03.178865 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 700/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 12:43:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:43:03.331887 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1d51d-b18b-465e-8a78-8aaabea8b7c5 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:43:03.333435 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-5af1d51d-b18b-465e-8a78-8aaabea8b7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:03.417053 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:43:03.479509 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1d51d-b18b-465e-8a78-8aaabea8b7c5 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:43:03.590764 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962b813b-4c2f-4d0e-8fa8-fb64574cf834 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:43:03.591289 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b813b-4c2f-4d0e-8fa8-fb64574cf834 service nova] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:03.591662 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b813b-4c2f-4d0e-8fa8-fb64574cf834 service nova] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:03.591928 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b813b-4c2f-4d0e-8fa8-fb64574cf834 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:03.593494 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-962b813b-4c2f-4d0e-8fa8-fb64574cf834 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:03.642478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:43:03.684846 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962b813b-4c2f-4d0e-8fa8-fb64574cf834 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:43:03.686465 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962b813b-4c2f-4d0e-8fa8-fb64574cf834 service nova] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:43:03.704501 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 728/1428] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:43:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:43:04.867342 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c5db0-a29c-4321-a526-a353ef647f0b None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:04.867890 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c5db0-a29c-4321-a526-a353ef647f0b None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:04.867890 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c5db0-a29c-4321-a526-a353ef647f0b None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:04.869241 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 701/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 12:43:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:43:04.876244 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b291c-5f6a-415e-a444-aa13852e4398 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:04.876670 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b291c-5f6a-415e-a444-aa13852e4398 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:04.876807 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b291c-5f6a-415e-a444-aa13852e4398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:04.951693 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-db5b291c-5f6a-415e-a444-aa13852e4398 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:43:04.952907 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-db5b291c-5f6a-415e-a444-aa13852e4398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:05.047850 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 729/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 12:43:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:43:05.200026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5c5db0-a29c-4321-a526-a353ef647f0b None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:43:05.201440 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-0f5c5db0-a29c-4321-a526-a353ef647f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:05.250869 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:43:05.349572 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5c5db0-a29c-4321-a526-a353ef647f0b None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:43:05.464792 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ac536-e9b2-436d-9c71-a9ee9cf31915 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:43:05.465330 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ac536-e9b2-436d-9c71-a9ee9cf31915 service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:05.465630 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ac536-e9b2-436d-9c71-a9ee9cf31915 service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:05.465901 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ac536-e9b2-436d-9c71-a9ee9cf31915 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:05.467535 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-8d5ac536-e9b2-436d-9c71-a9ee9cf31915 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:05.475493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:43:05.556129 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5ac536-e9b2-436d-9c71-a9ee9cf31915 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:43:05.557710 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5ac536-e9b2-436d-9c71-a9ee9cf31915 service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:43:05.575723 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 702/1431] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:43:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:43:06.734315 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85266425-7754-4e7f-b33e-0d25c970df13 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:06.734886 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85266425-7754-4e7f-b33e-0d25c970df13 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:06.734886 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85266425-7754-4e7f-b33e-0d25c970df13 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:06.736949 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 730/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 12:43:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:43:06.743183 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c9e6e-a27e-4594-8e22-a9b55cfad735 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:06.743667 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c9e6e-a27e-4594-8e22-a9b55cfad735 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:06.743779 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c9e6e-a27e-4594-8e22-a9b55cfad735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:06.819769 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-685c9e6e-a27e-4594-8e22-a9b55cfad735 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:43:06.820962 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-685c9e6e-a27e-4594-8e22-a9b55cfad735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:06.887793 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:43:06.915991 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 703/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 12:43:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:43:07.074213 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85266425-7754-4e7f-b33e-0d25c970df13 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:43:07.075762 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-85266425-7754-4e7f-b33e-0d25c970df13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:07.093097 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:43:07.221249 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85266425-7754-4e7f-b33e-0d25c970df13 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:43:07.309556 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:43:07.335240 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5008f7ac-08cd-4e18-b0a4-68246a2d5aa8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:43:07.335806 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008f7ac-08cd-4e18-b0a4-68246a2d5aa8 service nova] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:07.336081 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008f7ac-08cd-4e18-b0a4-68246a2d5aa8 service nova] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:07.336613 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008f7ac-08cd-4e18-b0a4-68246a2d5aa8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:07.338173 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-5008f7ac-08cd-4e18-b0a4-68246a2d5aa8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:07.427928 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5008f7ac-08cd-4e18-b0a4-68246a2d5aa8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:43:07.429517 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5008f7ac-08cd-4e18-b0a4-68246a2d5aa8 service nova] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:43:07.447629 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 731/1434] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:43:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:43:08.570826 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b579b-929b-4f24-9d62-880440718abc None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:08.571383 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b579b-929b-4f24-9d62-880440718abc None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:08.571383 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b579b-929b-4f24-9d62-880440718abc None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:08.572785 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 704/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 12:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:43:08.579217 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0dc40-153f-4aa2-8d17-90185ff2f4c2 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:08.579710 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0dc40-153f-4aa2-8d17-90185ff2f4c2 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:08.579810 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0dc40-153f-4aa2-8d17-90185ff2f4c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:08.654627 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-64e0dc40-153f-4aa2-8d17-90185ff2f4c2 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:43:08.655776 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-64e0dc40-153f-4aa2-8d17-90185ff2f4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:08.697168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:43:08.751502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 732/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 12:43:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:43:08.900882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b579b-929b-4f24-9d62-880440718abc None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:43:08.902425 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-fb7b579b-929b-4f24-9d62-880440718abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:09.044434 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b579b-929b-4f24-9d62-880440718abc None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:43:09.098827 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:43:09.156878 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d69ff-4c22-43ce-97ec-fe788dd76487 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:43:09.158286 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d69ff-4c22-43ce-97ec-fe788dd76487 service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:09.158558 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d69ff-4c22-43ce-97ec-fe788dd76487 service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:09.158827 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d69ff-4c22-43ce-97ec-fe788dd76487 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:09.160443 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-695d69ff-4c22-43ce-97ec-fe788dd76487 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:09.252845 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d69ff-4c22-43ce-97ec-fe788dd76487 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:43:09.254474 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d69ff-4c22-43ce-97ec-fe788dd76487 service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:43:09.273516 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 705/1437] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:43:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:43:10.422339 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30bfe4d-ea68-4f0b-aaf9-318192b5198a None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:10.422879 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30bfe4d-ea68-4f0b-aaf9-318192b5198a None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:10.422879 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30bfe4d-ea68-4f0b-aaf9-318192b5198a None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:10.424323 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 733/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 12:43:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:43:10.430698 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930acfe-97cf-4f41-b561-b0a708313c9f None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:10.431103 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930acfe-97cf-4f41-b561-b0a708313c9f None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:10.431254 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930acfe-97cf-4f41-b561-b0a708313c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:10.512474 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-5930acfe-97cf-4f41-b561-b0a708313c9f None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:43:10.513678 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-5930acfe-97cf-4f41-b561-b0a708313c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:10.515683 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:43:10.610080 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 706/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 12:43:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:43:10.758599 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30bfe4d-ea68-4f0b-aaf9-318192b5198a None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:43:10.760178 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-a30bfe4d-ea68-4f0b-aaf9-318192b5198a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:10.905435 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30bfe4d-ea68-4f0b-aaf9-318192b5198a None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:43:10.931713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:43:11.015791 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d76754-d686-4cde-b935-fe0d804d6121 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:43:11.016421 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d76754-d686-4cde-b935-fe0d804d6121 service nova] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:11.017226 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d76754-d686-4cde-b935-fe0d804d6121 service nova] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:11.017575 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d76754-d686-4cde-b935-fe0d804d6121 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:11.019160 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-75d76754-d686-4cde-b935-fe0d804d6121 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:11.106752 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d76754-d686-4cde-b935-fe0d804d6121 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:43:11.108353 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d76754-d686-4cde-b935-fe0d804d6121 service nova] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:43:11.126775 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 734/1440] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:43:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:43:13.191187 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b9d09-b7fd-40c2-a167-3e2cd977d572 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:13.191833 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b9d09-b7fd-40c2-a167-3e2cd977d572 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:13.191833 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b9d09-b7fd-40c2-a167-3e2cd977d572 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:13.193136 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 707/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 12:43:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:43:13.198680 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e692179-e921-45cb-8ba1-8e4779117033 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:13.199109 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e692179-e921-45cb-8ba1-8e4779117033 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:13.199210 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e692179-e921-45cb-8ba1-8e4779117033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:13.241048 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:43:13.266668 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-6e692179-e921-45cb-8ba1-8e4779117033 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:43:13.267910 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-6e692179-e921-45cb-8ba1-8e4779117033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:13.365022 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 735/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 12:43:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:43:13.646536 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b9d09-b7fd-40c2-a167-3e2cd977d572 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:43:13.648120 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-587b9d09-b7fd-40c2-a167-3e2cd977d572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:13.753423 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b9d09-b7fd-40c2-a167-3e2cd977d572 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:43:13.785343 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:43:13.825713 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee8df5-e1ab-40b7-b103-ea072cb4a277 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:43:13.826328 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee8df5-e1ab-40b7-b103-ea072cb4a277 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:13.826639 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee8df5-e1ab-40b7-b103-ea072cb4a277 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:13.826941 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee8df5-e1ab-40b7-b103-ea072cb4a277 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:13.828562 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-08ee8df5-e1ab-40b7-b103-ea072cb4a277 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:13.936230 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee8df5-e1ab-40b7-b103-ea072cb4a277 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:43:13.937590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee8df5-e1ab-40b7-b103-ea072cb4a277 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:43:13.956886 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 708/1443] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:43:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:43:16.304147 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad64130-5eea-427a-9a9f-45f71d60ab9c None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:16.304674 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad64130-5eea-427a-9a9f-45f71d60ab9c None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:16.304674 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad64130-5eea-427a-9a9f-45f71d60ab9c None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:16.306014 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 736/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 12:43:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:43:16.311139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e41eeb-f861-4bda-9eb1-0e58edd57867 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:16.311587 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e41eeb-f861-4bda-9eb1-0e58edd57867 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:16.311713 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e41eeb-f861-4bda-9eb1-0e58edd57867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:16.379487 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-73e41eeb-f861-4bda-9eb1-0e58edd57867 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:43:16.380612 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-73e41eeb-f861-4bda-9eb1-0e58edd57867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:16.393872 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:43:16.477388 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 709/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 12:43:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:43:16.765484 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad64130-5eea-427a-9a9f-45f71d60ab9c None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:43:16.767724 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-aad64130-5eea-427a-9a9f-45f71d60ab9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:16.868942 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad64130-5eea-427a-9a9f-45f71d60ab9c None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:43:16.882046 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:43:16.941871 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6560ac-bd52-4828-bbd4-8c1b87b4df29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:43:16.942474 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6560ac-bd52-4828-bbd4-8c1b87b4df29 service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:16.942746 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6560ac-bd52-4828-bbd4-8c1b87b4df29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:16.943014 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6560ac-bd52-4828-bbd4-8c1b87b4df29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:16.944597 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-3e6560ac-bd52-4828-bbd4-8c1b87b4df29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:17.049371 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6560ac-bd52-4828-bbd4-8c1b87b4df29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:43:17.050714 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6560ac-bd52-4828-bbd4-8c1b87b4df29 service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:43:17.068437 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 737/1446] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:43:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:43:19.621704 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7daf6-a5c3-4566-b421-34d1da2bf340 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:19.622248 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7daf6-a5c3-4566-b421-34d1da2bf340 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:19.622248 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7daf6-a5c3-4566-b421-34d1da2bf340 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:19.623635 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 710/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 12:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:43:19.629100 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb1028-c1b1-4e9c-b299-2469521de9e1 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:19.629525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb1028-c1b1-4e9c-b299-2469521de9e1 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:19.629633 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb1028-c1b1-4e9c-b299-2469521de9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:19.676963 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:43:19.694424 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-c1cb1028-c1b1-4e9c-b299-2469521de9e1 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:43:19.695654 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-c1cb1028-c1b1-4e9c-b299-2469521de9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:19.790454 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 738/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 12:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:43:20.087936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7daf6-a5c3-4566-b421-34d1da2bf340 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:43:20.089345 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-c3a7daf6-a5c3-4566-b421-34d1da2bf340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:20.191656 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7daf6-a5c3-4566-b421-34d1da2bf340 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:43:20.231799 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:43:20.268461 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcb604c-07e4-412b-88ae-909b292f34b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:43:20.269075 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb604c-07e4-412b-88ae-909b292f34b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:20.269351 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb604c-07e4-412b-88ae-909b292f34b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:20.269632 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb604c-07e4-412b-88ae-909b292f34b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:20.271176 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-cdcb604c-07e4-412b-88ae-909b292f34b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:20.380946 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcb604c-07e4-412b-88ae-909b292f34b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:43:20.382300 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcb604c-07e4-412b-88ae-909b292f34b6 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:43:20.400473 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 711/1449] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:43:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:43:22.667013 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd19bb9c-6538-4da9-a06f-82168a4084c1 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:22.667605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd19bb9c-6538-4da9-a06f-82168a4084c1 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:22.667605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd19bb9c-6538-4da9-a06f-82168a4084c1 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:22.668917 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 739/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 12:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:43:22.674742 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea55c37-adb1-440d-936d-5c4352d58007 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:22.675219 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea55c37-adb1-440d-936d-5c4352d58007 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:22.675357 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea55c37-adb1-440d-936d-5c4352d58007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:22.740154 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-0ea55c37-adb1-440d-936d-5c4352d58007 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:43:22.741334 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-0ea55c37-adb1-440d-936d-5c4352d58007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:22.761502 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:43:22.836467 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 712/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 12:43:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:43:23.131801 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd19bb9c-6538-4da9-a06f-82168a4084c1 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:43:23.133327 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-cd19bb9c-6538-4da9-a06f-82168a4084c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:23.236915 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd19bb9c-6538-4da9-a06f-82168a4084c1 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:43:23.258605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:43:23.313122 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357b7b08-84e9-48cc-bb76-8da2b67c8110 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:43:23.313762 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b7b08-84e9-48cc-bb76-8da2b67c8110 service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:23.314042 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b7b08-84e9-48cc-bb76-8da2b67c8110 service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:23.314311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b7b08-84e9-48cc-bb76-8da2b67c8110 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:23.315927 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-357b7b08-84e9-48cc-bb76-8da2b67c8110 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:23.422793 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357b7b08-84e9-48cc-bb76-8da2b67c8110 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:43:23.424251 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357b7b08-84e9-48cc-bb76-8da2b67c8110 service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:43:23.441912 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 740/1452] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:43:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:43:25.648494 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e898eb37-fb76-4020-9e8d-0596adf45f4f None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:25.649033 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e898eb37-fb76-4020-9e8d-0596adf45f4f None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:25.649033 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e898eb37-fb76-4020-9e8d-0596adf45f4f None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:25.650397 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 713/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 12:43:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:43:25.655617 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d20c68-6ba8-43c3-b214-d2b9420879b5 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:25.656036 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d20c68-6ba8-43c3-b214-d2b9420879b5 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:25.656156 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d20c68-6ba8-43c3-b214-d2b9420879b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:25.722012 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:43:25.724935 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-51d20c68-6ba8-43c3-b214-d2b9420879b5 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:43:25.730985 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-51d20c68-6ba8-43c3-b214-d2b9420879b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:25.823956 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 741/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 12:43:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:43:26.143956 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e898eb37-fb76-4020-9e8d-0596adf45f4f None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:43:26.145484 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e898eb37-fb76-4020-9e8d-0596adf45f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:26.254911 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e898eb37-fb76-4020-9e8d-0596adf45f4f None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:43:26.305998 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:43:26.331832 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4c06df-a968-49c6-97cc-62278645c8d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:43:26.332445 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c06df-a968-49c6-97cc-62278645c8d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:26.332722 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c06df-a968-49c6-97cc-62278645c8d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:26.332991 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c06df-a968-49c6-97cc-62278645c8d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:26.334565 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4d4c06df-a968-49c6-97cc-62278645c8d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:26.447456 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4c06df-a968-49c6-97cc-62278645c8d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:43:26.448802 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4c06df-a968-49c6-97cc-62278645c8d6 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:43:26.466272 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 714/1455] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:43:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:43:28.691250 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed378803-9f82-4bb8-b62b-ca22d1b74e64 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:28.691865 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed378803-9f82-4bb8-b62b-ca22d1b74e64 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:28.691865 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed378803-9f82-4bb8-b62b-ca22d1b74e64 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:28.693180 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 742/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 12:43:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:43:28.698696 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626fbcc-d189-4a9d-978f-3c3ba2c37b80 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:28.699117 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626fbcc-d189-4a9d-978f-3c3ba2c37b80 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:28.699252 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626fbcc-d189-4a9d-978f-3c3ba2c37b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:28.767078 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-0626fbcc-d189-4a9d-978f-3c3ba2c37b80 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:43:28.768322 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-0626fbcc-d189-4a9d-978f-3c3ba2c37b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:28.796656 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:43:28.863725 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 715/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 12:43:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:43:29.148407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed378803-9f82-4bb8-b62b-ca22d1b74e64 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:43:29.149925 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-ed378803-9f82-4bb8-b62b-ca22d1b74e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:29.255147 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed378803-9f82-4bb8-b62b-ca22d1b74e64 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:43:29.283436 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:43:29.328693 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced2d2da-2d1e-414c-9721-4a0b10de8272 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:43:29.329320 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2d2da-2d1e-414c-9721-4a0b10de8272 service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:29.329599 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2d2da-2d1e-414c-9721-4a0b10de8272 service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:29.329862 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2d2da-2d1e-414c-9721-4a0b10de8272 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:29.331470 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-ced2d2da-2d1e-414c-9721-4a0b10de8272 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:29.440026 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced2d2da-2d1e-414c-9721-4a0b10de8272 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:43:29.441457 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced2d2da-2d1e-414c-9721-4a0b10de8272 service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:43:29.459662 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 743/1458] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:43:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:43:31.796344 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc36307-cbe1-4988-af13-8daee41b9c51 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:31.796911 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc36307-cbe1-4988-af13-8daee41b9c51 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:31.796911 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc36307-cbe1-4988-af13-8daee41b9c51 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:31.798288 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 716/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 12:43:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:43:31.803716 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1584dc8b-513b-43e3-b6f4-c4b8874c03d5 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:31.804148 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1584dc8b-513b-43e3-b6f4-c4b8874c03d5 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:31.804279 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1584dc8b-513b-43e3-b6f4-c4b8874c03d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:31.870020 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-1584dc8b-513b-43e3-b6f4-c4b8874c03d5 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:43:31.871238 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-1584dc8b-513b-43e3-b6f4-c4b8874c03d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:31.880192 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:43:31.967781 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 744/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 12:43:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:43:32.258602 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc36307-cbe1-4988-af13-8daee41b9c51 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:43:32.260188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-6bc36307-cbe1-4988-af13-8daee41b9c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:32.365000 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc36307-cbe1-4988-af13-8daee41b9c51 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:43:32.421718 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:43:32.439814 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc42fc-5ba4-4cf2-b646-acff706dc921 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:43:32.440993 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc42fc-5ba4-4cf2-b646-acff706dc921 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:32.441268 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc42fc-5ba4-4cf2-b646-acff706dc921 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:32.441537 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc42fc-5ba4-4cf2-b646-acff706dc921 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:32.443122 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-32cc42fc-5ba4-4cf2-b646-acff706dc921 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:32.553002 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc42fc-5ba4-4cf2-b646-acff706dc921 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:43:32.554432 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc42fc-5ba4-4cf2-b646-acff706dc921 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:43:32.572579 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 717/1461] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:43:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:43:34.764971 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987f36d-eeef-4f7a-a3aa-827528f00728 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:34.765546 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987f36d-eeef-4f7a-a3aa-827528f00728 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:34.765546 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987f36d-eeef-4f7a-a3aa-827528f00728 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:34.767525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 745/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 12:43:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:43:34.772929 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b2270-d2be-4271-9aca-f0025bcb8e7b None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:34.773345 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b2270-d2be-4271-9aca-f0025bcb8e7b None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:34.773539 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b2270-d2be-4271-9aca-f0025bcb8e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:34.838557 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-3f0b2270-d2be-4271-9aca-f0025bcb8e7b None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:43:34.839794 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-3f0b2270-d2be-4271-9aca-f0025bcb8e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:34.877964 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:43:34.934647 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 718/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 12:43:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:43:35.220511 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a987f36d-eeef-4f7a-a3aa-827528f00728 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:43:35.222040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-a987f36d-eeef-4f7a-a3aa-827528f00728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:35.327521 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a987f36d-eeef-4f7a-a3aa-827528f00728 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:43:35.375065 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:43:35.420624 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eabd29-4b1e-4397-8de9-af723c27eae5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:43:35.421237 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eabd29-4b1e-4397-8de9-af723c27eae5 service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:35.421554 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eabd29-4b1e-4397-8de9-af723c27eae5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:35.421826 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eabd29-4b1e-4397-8de9-af723c27eae5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:35.424105 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-39eabd29-4b1e-4397-8de9-af723c27eae5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:35.552154 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eabd29-4b1e-4397-8de9-af723c27eae5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:43:35.553597 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eabd29-4b1e-4397-8de9-af723c27eae5 service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:43:35.576226 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 746/1464] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:43:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:43:37.783572 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d928af7-9115-4d56-9ad1-a05a1028f503 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:37.784141 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d928af7-9115-4d56-9ad1-a05a1028f503 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:37.784141 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d928af7-9115-4d56-9ad1-a05a1028f503 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:37.785464 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 719/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 12:43:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:43:37.790776 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4e240-ea07-4065-b3fa-085bbe6960cd None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:37.791211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4e240-ea07-4065-b3fa-085bbe6960cd None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:37.791349 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4e240-ea07-4065-b3fa-085bbe6960cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:37.858195 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-0ff4e240-ea07-4065-b3fa-085bbe6960cd None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:43:37.859412 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-0ff4e240-ea07-4065-b3fa-085bbe6960cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:37.878641 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:43:37.956409 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 747/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 12:43:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:43:38.240936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d928af7-9115-4d56-9ad1-a05a1028f503 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:43:38.242460 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-5d928af7-9115-4d56-9ad1-a05a1028f503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:38.349242 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d928af7-9115-4d56-9ad1-a05a1028f503 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:43:38.424204 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:43:38.428743 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d1292f-f549-4dcd-8006-f078a1eebdcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:43:38.429364 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1292f-f549-4dcd-8006-f078a1eebdcf service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:38.429640 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1292f-f549-4dcd-8006-f078a1eebdcf service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:38.429906 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1292f-f549-4dcd-8006-f078a1eebdcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:38.431542 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-42d1292f-f549-4dcd-8006-f078a1eebdcf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:38.547171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d1292f-f549-4dcd-8006-f078a1eebdcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:43:38.549679 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d1292f-f549-4dcd-8006-f078a1eebdcf service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:43:38.568989 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 720/1467] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:43:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:43:40.736517 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e11613-91dd-4f77-bde9-832b7a2d7847 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:40.737083 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e11613-91dd-4f77-bde9-832b7a2d7847 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:40.737083 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e11613-91dd-4f77-bde9-832b7a2d7847 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:40.738464 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 748/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 12:43:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:43:40.743850 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa91592-88fb-42e8-b837-0e06267e9930 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:40.744307 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa91592-88fb-42e8-b837-0e06267e9930 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:40.744416 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa91592-88fb-42e8-b837-0e06267e9930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:40.809133 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-efa91592-88fb-42e8-b837-0e06267e9930 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:43:40.810326 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-efa91592-88fb-42e8-b837-0e06267e9930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:40.857662 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:43:40.905153 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 721/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 12:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:43:41.187479 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e11613-91dd-4f77-bde9-832b7a2d7847 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:43:41.189880 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-51e11613-91dd-4f77-bde9-832b7a2d7847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:41.296502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e11613-91dd-4f77-bde9-832b7a2d7847 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:43:41.345196 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:43:41.374071 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784f1b53-f00c-4cbc-af8f-bb88d2b8869a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:43:41.374683 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f1b53-f00c-4cbc-af8f-bb88d2b8869a service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:43:41.374962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f1b53-f00c-4cbc-af8f-bb88d2b8869a service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:43:41.375229 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f1b53-f00c-4cbc-af8f-bb88d2b8869a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:43:41.377434 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-784f1b53-f00c-4cbc-af8f-bb88d2b8869a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:43:41.487703 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784f1b53-f00c-4cbc-af8f-bb88d2b8869a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:43:41.489056 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784f1b53-f00c-4cbc-af8f-bb88d2b8869a service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:43:41.507693 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 749/1470] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:43:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:43:50.903595 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=22 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:43:51.505574 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=66 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:44:01.985427 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6332d9de-e345-4e76-afa4-39616f008496 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:44:01.986059 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6332d9de-e345-4e76-afa4-39616f008496 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:44:01.986059 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6332d9de-e345-4e76-afa4-39616f008496 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:44:01.987405 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 722/1471] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:44:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:44:01.992851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2cd9e-3511-424e-ac5a-33e27ef10cbe None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:44:01.993311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2cd9e-3511-424e-ac5a-33e27ef10cbe None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:44:01.993496 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2cd9e-3511-424e-ac5a-33e27ef10cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:44:02.060757 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-5db2cd9e-3511-424e-ac5a-33e27ef10cbe None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:44:02.062039 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-5db2cd9e-3511-424e-ac5a-33e27ef10cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:44:02.162393 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 750/1472] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:44:02.527616 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591727e-31ef-41cf-b1c2-87238fcdf30a None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:44:02.527935 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591727e-31ef-41cf-b1c2-87238fcdf30a None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:44:02.528164 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591727e-31ef-41cf-b1c2-87238fcdf30a None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:44:02.529488 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 723/1473] 10.222.0.33 () {54 vars in 962 bytes} [Mon Oct 3 12:44:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:44:02.534432 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8dcb75-b967-4814-b6a8-1f12f433431d None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:44:02.534710 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8dcb75-b967-4814-b6a8-1f12f433431d None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:44:02.535001 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8dcb75-b967-4814-b6a8-1f12f433431d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:44:02.610576 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-0e8dcb75-b967-4814-b6a8-1f12f433431d None None] MFA Rules not processed for user `b697e4847b13496d98a543675f5df1c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:44:02.611923 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-0e8dcb75-b967-4814-b6a8-1f12f433431d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:44:02.632021 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:44:02.687429 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 751/1474] 10.222.0.33 () {58 vars in 1059 bytes} [Mon Oct 3 12:44:02 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:44:02.695308 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a591727e-31ef-41cf-b1c2-87238fcdf30a None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:44:02.696869 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a591727e-31ef-41cf-b1c2-87238fcdf30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:44:02.801269 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a591727e-31ef-41cf-b1c2-87238fcdf30a None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:44:02.875305 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586fa5f6-912e-4bbd-b15d-249966228f35 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b697e4847b13496d98a543675f5df1c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:44:02.875940 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586fa5f6-912e-4bbd-b15d-249966228f35 service glance] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:44:02.876217 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586fa5f6-912e-4bbd-b15d-249966228f35 service glance] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:44:02.876488 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586fa5f6-912e-4bbd-b15d-249966228f35 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:44:02.878099 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-586fa5f6-912e-4bbd-b15d-249966228f35 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:44:02.933512 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:44:02.994341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586fa5f6-912e-4bbd-b15d-249966228f35 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:44:02.996105 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586fa5f6-912e-4bbd-b15d-249966228f35 service glance] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:44:03.018804 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 724/1475] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:44:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:44:05.248577 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53860652-b353-4eca-93cd-5fb5a4b29e37 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:44:05.249137 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53860652-b353-4eca-93cd-5fb5a4b29e37 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:44:05.249137 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53860652-b353-4eca-93cd-5fb5a4b29e37 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:44:05.250438 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 752/1476] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:44:05.256417 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d3021-f30c-4b79-aeaf-15374bf462ea None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:44:05.256837 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d3021-f30c-4b79-aeaf-15374bf462ea None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:44:05.256947 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d3021-f30c-4b79-aeaf-15374bf462ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:44:05.323274 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-8e1d3021-f30c-4b79-aeaf-15374bf462ea None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:44:05.324489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-8e1d3021-f30c-4b79-aeaf-15374bf462ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:44:05.420420 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 725/1477] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:44:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:44:05.691965 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53860652-b353-4eca-93cd-5fb5a4b29e37 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:44:05.693392 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-53860652-b353-4eca-93cd-5fb5a4b29e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:44:05.823626 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53860652-b353-4eca-93cd-5fb5a4b29e37 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:44:05.873335 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:44:05.918956 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150c2a3-37b0-4c06-933f-138fb5d90549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:44:05.919592 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150c2a3-37b0-4c06-933f-138fb5d90549 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:44:05.919868 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150c2a3-37b0-4c06-933f-138fb5d90549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:44:05.920136 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150c2a3-37b0-4c06-933f-138fb5d90549 admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:44:05.920764 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150c2a3-37b0-4c06-933f-138fb5d90549 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:44:05.922467 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: WARNING py.warnings [None req-3150c2a3-37b0-4c06-933f-138fb5d90549 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 12:44:05.922467 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: warnings.warn(msg) Oct 03 12:44:05.922467 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: Oct 03 12:44:05.924323 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150c2a3-37b0-4c06-933f-138fb5d90549 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:44:05.942940 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 753/1478] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 12:44:05 2022] GET /identity/v3/projects => generated 2553 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:44:08.135800 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd660bf-9d01-499d-b341-cb0d67d664ad None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:44:08.136234 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd660bf-9d01-499d-b341-cb0d67d664ad None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:44:08.136345 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd660bf-9d01-499d-b341-cb0d67d664ad None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:44:08.137675 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 726/1479] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:44:08.143831 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973cde7d-5bea-49d7-a165-f06c1dd6eca0 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:44:08.144258 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973cde7d-5bea-49d7-a165-f06c1dd6eca0 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:44:08.144387 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973cde7d-5bea-49d7-a165-f06c1dd6eca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:44:08.214848 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-973cde7d-5bea-49d7-a165-f06c1dd6eca0 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:44:08.216108 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-973cde7d-5bea-49d7-a165-f06c1dd6eca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:44:08.312774 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 754/1480] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:44:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:44:08.605371 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd660bf-9d01-499d-b341-cb0d67d664ad None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:44:08.606873 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-0bd660bf-9d01-499d-b341-cb0d67d664ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:44:08.617171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:44:08.751514 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd660bf-9d01-499d-b341-cb0d67d664ad None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:44:08.831229 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:44:08.864567 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3aa17-350c-44bf-8674-1f8c04cbdee0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:44:08.864567 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3aa17-350c-44bf-8674-1f8c04cbdee0 service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:44:08.864567 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3aa17-350c-44bf-8674-1f8c04cbdee0 service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:44:08.864964 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3aa17-350c-44bf-8674-1f8c04cbdee0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:44:08.866306 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ccd3aa17-350c-44bf-8674-1f8c04cbdee0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:44:08.960794 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd3aa17-350c-44bf-8674-1f8c04cbdee0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:44:08.962380 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd3aa17-350c-44bf-8674-1f8c04cbdee0 service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:44:08.981129 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 727/1481] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:44:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:44:11.378413 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c951d9-c278-4c74-ad13-28efb1801730 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:44:11.378945 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c951d9-c278-4c74-ad13-28efb1801730 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:44:11.378945 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c951d9-c278-4c74-ad13-28efb1801730 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:44:11.380345 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 755/1482] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:44:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:44:11.385797 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829139b-c5f9-4778-b0df-af991a49fb9b None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:44:11.386247 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829139b-c5f9-4778-b0df-af991a49fb9b None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:44:11.386345 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829139b-c5f9-4778-b0df-af991a49fb9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:44:11.451504 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-1829139b-c5f9-4778-b0df-af991a49fb9b None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:44:11.452688 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1829139b-c5f9-4778-b0df-af991a49fb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:44:11.497946 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:44:11.547600 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 728/1483] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:44:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:44:11.838392 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c951d9-c278-4c74-ad13-28efb1801730 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:44:11.839944 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-e5c951d9-c278-4c74-ad13-28efb1801730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:44:11.864305 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:44:11.944162 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c951d9-c278-4c74-ad13-28efb1801730 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:44:12.016555 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb5be2f-da86-41e9-b0ec-adec1355b5c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:44:12.017162 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5be2f-da86-41e9-b0ec-adec1355b5c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:44:12.017443 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5be2f-da86-41e9-b0ec-adec1355b5c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:44:12.017743 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5be2f-da86-41e9-b0ec-adec1355b5c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:44:12.019374 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-5bb5be2f-da86-41e9-b0ec-adec1355b5c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:44:12.092208 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:44:12.126405 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb5be2f-da86-41e9-b0ec-adec1355b5c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:44:12.128086 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb5be2f-da86-41e9-b0ec-adec1355b5c1 service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:44:12.145875 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 756/1484] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:44:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:44:14.391500 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f96495-55c9-4aff-9f3e-50599d869b93 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:44:14.392108 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f96495-55c9-4aff-9f3e-50599d869b93 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:44:14.392108 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f96495-55c9-4aff-9f3e-50599d869b93 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:44:14.393985 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 729/1485] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:44:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:44:14.399230 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7cd0c-1dab-441d-9300-ff1670477596 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:44:14.399707 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7cd0c-1dab-441d-9300-ff1670477596 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:44:14.399823 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7cd0c-1dab-441d-9300-ff1670477596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:44:14.466841 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-e9a7cd0c-1dab-441d-9300-ff1670477596 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:44:14.468074 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-e9a7cd0c-1dab-441d-9300-ff1670477596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:44:14.563147 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 757/1486] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:44:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:44:14.855524 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f96495-55c9-4aff-9f3e-50599d869b93 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:44:14.857071 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b5f96495-55c9-4aff-9f3e-50599d869b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:44:14.959442 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f96495-55c9-4aff-9f3e-50599d869b93 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:44:15.031422 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:44:15.033500 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd87bd6f-cebc-4e09-b3f4-6ef41913f89c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:44:15.034106 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87bd6f-cebc-4e09-b3f4-6ef41913f89c service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:44:15.034379 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87bd6f-cebc-4e09-b3f4-6ef41913f89c service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:44:15.034646 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87bd6f-cebc-4e09-b3f4-6ef41913f89c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:44:15.036281 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-bd87bd6f-cebc-4e09-b3f4-6ef41913f89c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:44:15.142838 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd87bd6f-cebc-4e09-b3f4-6ef41913f89c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:44:15.144220 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd87bd6f-cebc-4e09-b3f4-6ef41913f89c service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:44:15.163184 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 730/1487] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:44:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:44:17.413917 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbbefe-45d8-49ef-8a9a-9bd10bf2f953 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:44:17.414470 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbbefe-45d8-49ef-8a9a-9bd10bf2f953 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:44:17.414470 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbbefe-45d8-49ef-8a9a-9bd10bf2f953 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:44:17.415895 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 758/1488] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:44:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:44:17.421080 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59aae0-c26c-44a4-92d1-fdf9ad47c699 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:44:17.421507 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59aae0-c26c-44a4-92d1-fdf9ad47c699 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:44:17.421655 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59aae0-c26c-44a4-92d1-fdf9ad47c699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:44:17.488428 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-cb59aae0-c26c-44a4-92d1-fdf9ad47c699 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:44:17.489624 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-cb59aae0-c26c-44a4-92d1-fdf9ad47c699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:44:17.541962 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:44:17.585317 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 731/1489] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:44:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:44:17.875015 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcbbefe-45d8-49ef-8a9a-9bd10bf2f953 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:44:17.876638 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-efcbbefe-45d8-49ef-8a9a-9bd10bf2f953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:44:17.912749 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:44:17.981006 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcbbefe-45d8-49ef-8a9a-9bd10bf2f953 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:44:18.053112 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9e96b3-df1b-46cb-8cf1-239742cbdcfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:44:18.053720 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e96b3-df1b-46cb-8cf1-239742cbdcfe service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:44:18.054002 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e96b3-df1b-46cb-8cf1-239742cbdcfe service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:44:18.054364 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e96b3-df1b-46cb-8cf1-239742cbdcfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:44:18.055911 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-cf9e96b3-df1b-46cb-8cf1-239742cbdcfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:44:18.137099 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:44:18.166343 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9e96b3-df1b-46cb-8cf1-239742cbdcfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:44:18.167849 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9e96b3-df1b-46cb-8cf1-239742cbdcfe service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:44:18.186625 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 759/1490] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:44:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:44:20.879988 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72fab67-d86f-466d-b91c-422064fe2528 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:44:20.880554 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72fab67-d86f-466d-b91c-422064fe2528 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:44:20.880554 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72fab67-d86f-466d-b91c-422064fe2528 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:44:20.881915 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 732/1491] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 12:44:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:44:20.887367 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e30a60-5a14-4944-accd-8ee9e290c479 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:44:20.887794 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e30a60-5a14-4944-accd-8ee9e290c479 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:44:20.887932 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e30a60-5a14-4944-accd-8ee9e290c479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:44:20.953815 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-c1e30a60-5a14-4944-accd-8ee9e290c479 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:44:20.955661 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-c1e30a60-5a14-4944-accd-8ee9e290c479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:44:21.049548 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 760/1492] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 12:44:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:44:21.344043 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72fab67-d86f-466d-b91c-422064fe2528 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:44:21.345790 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d72fab67-d86f-466d-b91c-422064fe2528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:44:21.450887 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72fab67-d86f-466d-b91c-422064fe2528 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:44:21.525351 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04889310-5d8c-446b-b687-3e962d93f5b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:44:21.525881 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04889310-5d8c-446b-b687-3e962d93f5b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:44:21.526158 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04889310-5d8c-446b-b687-3e962d93f5b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:44:21.526428 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04889310-5d8c-446b-b687-3e962d93f5b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:44:21.528034 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-04889310-5d8c-446b-b687-3e962d93f5b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:44:21.536613 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:44:21.640598 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04889310-5d8c-446b-b687-3e962d93f5b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:44:21.642026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04889310-5d8c-446b-b687-3e962d93f5b7 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:44:21.660983 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 733/1493] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:44:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:44:31.047831 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=82 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:44:31.658031 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=50 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:45:50.161525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a4f8c4-6a38-4d1e-b528-85a04e8f2ad9 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:45:50.162126 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a4f8c4-6a38-4d1e-b528-85a04e8f2ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:45:50.162126 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a4f8c4-6a38-4d1e-b528-85a04e8f2ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:45:50.240115 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-94a4f8c4-6a38-4d1e-b528-85a04e8f2ad9 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:45:50.242025 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-94a4f8c4-6a38-4d1e-b528-85a04e8f2ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:45:50.338009 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 761/1494] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:45:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:45:50.345998 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8c02ba-acaa-491a-9067-8843fec686e7 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:45:50.346480 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8c02ba-acaa-491a-9067-8843fec686e7 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:45:50.346600 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8c02ba-acaa-491a-9067-8843fec686e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:45:50.422046 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-ff8c02ba-acaa-491a-9067-8843fec686e7 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:45:50.423132 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ff8c02ba-acaa-491a-9067-8843fec686e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:45:50.520105 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 734/1495] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:45:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:45:50.524775 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a4f8c4-6a38-4d1e-b528-85a04e8f2ad9 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:45:50.526059 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-94a4f8c4-6a38-4d1e-b528-85a04e8f2ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:45:50.537444 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a4f8c4-6a38-4d1e-b528-85a04e8f2ad9 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:45:50.597983 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:45:50.630874 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594cf12f-a46d-410f-b06f-9c28dd65faf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:45:50.632070 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594cf12f-a46d-410f-b06f-9c28dd65faf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:45:50.632347 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594cf12f-a46d-410f-b06f-9c28dd65faf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:45:50.632622 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594cf12f-a46d-410f-b06f-9c28dd65faf3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:45:50.633235 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594cf12f-a46d-410f-b06f-9c28dd65faf3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:45:50.633535 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594cf12f-a46d-410f-b06f-9c28dd65faf3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:45:50.634534 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: WARNING py.warnings [None req-594cf12f-a46d-410f-b06f-9c28dd65faf3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 03 12:45:50.634534 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: warnings.warn(msg) Oct 03 12:45:50.634534 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: Oct 03 12:45:50.635437 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594cf12f-a46d-410f-b06f-9c28dd65faf3 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:45:50.646221 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 762/1496] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:45:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:45:50.654787 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8c02ba-acaa-491a-9067-8843fec686e7 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:45:50.663951 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8c02ba-acaa-491a-9067-8843fec686e7 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:45:50.725804 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:45:50.759553 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a30657-b629-4b2d-9436-f46cb5e13bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:45:50.760257 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a30657-b629-4b2d-9436-f46cb5e13bcb admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:45:50.760528 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a30657-b629-4b2d-9436-f46cb5e13bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:45:50.760790 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a30657-b629-4b2d-9436-f46cb5e13bcb admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:45:50.761413 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a30657-b629-4b2d-9436-f46cb5e13bcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:45:50.763111 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: WARNING py.warnings [None req-61a30657-b629-4b2d-9436-f46cb5e13bcb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 12:45:50.763111 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: warnings.warn(msg) Oct 03 12:45:50.763111 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: Oct 03 12:45:50.764380 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a30657-b629-4b2d-9436-f46cb5e13bcb admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:45:50.908186 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 735/1497] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:45:50 2022] POST /identity/v3/projects => generated 379 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:45:50.912433 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594cf12f-a46d-410f-b06f-9c28dd65faf3 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:45:50.914193 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-594cf12f-a46d-410f-b06f-9c28dd65faf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:45:50.948168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594cf12f-a46d-410f-b06f-9c28dd65faf3 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:45:51.050106 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b9d7e-a4c0-43e5-a6aa-29eedfbf941f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:45:51.050709 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b9d7e-a4c0-43e5-a6aa-29eedfbf941f admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:45:51.050985 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b9d7e-a4c0-43e5-a6aa-29eedfbf941f admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:45:51.051252 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b9d7e-a4c0-43e5-a6aa-29eedfbf941f admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:45:51.051905 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1b9d7e-a4c0-43e5-a6aa-29eedfbf941f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:45:51.052898 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: WARNING py.warnings [None req-7e1b9d7e-a4c0-43e5-a6aa-29eedfbf941f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 12:45:51.052898 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: warnings.warn(msg) Oct 03 12:45:51.052898 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: Oct 03 12:45:51.054090 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1b9d7e-a4c0-43e5-a6aa-29eedfbf941f admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:45:51.189685 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 763/1498] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:45:50 2022] POST /identity/v3/users => generated 290 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:45:51.193727 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a30657-b629-4b2d-9436-f46cb5e13bcb admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:45:51.195266 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-61a30657-b629-4b2d-9436-f46cb5e13bcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:45:51.260936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a30657-b629-4b2d-9436-f46cb5e13bcb admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:45:51.345525 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:45:51.356507 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deac7c75-3bb3-4270-a951-8ceef2acc70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:45:51.357099 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac7c75-3bb3-4270-a951-8ceef2acc70e admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:45:51.358470 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac7c75-3bb3-4270-a951-8ceef2acc70e admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:45:51.358470 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac7c75-3bb3-4270-a951-8ceef2acc70e admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:45:51.358470 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deac7c75-3bb3-4270-a951-8ceef2acc70e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:45:51.359283 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: WARNING py.warnings [None req-deac7c75-3bb3-4270-a951-8ceef2acc70e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 03 12:45:51.359283 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: warnings.warn(msg) Oct 03 12:45:51.359283 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: Oct 03 12:45:51.360223 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deac7c75-3bb3-4270-a951-8ceef2acc70e admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:45:51.373500 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 736/1499] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:45:51 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:45:51.377629 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b9d7e-a4c0-43e5-a6aa-29eedfbf941f admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:45:51.379178 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-7e1b9d7e-a4c0-43e5-a6aa-29eedfbf941f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:45:51.390978 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b9d7e-a4c0-43e5-a6aa-29eedfbf941f admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:45:51.416910 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:45:51.485107 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5973e076-24f8-4a72-a154-c5d4d2399805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:45:51.485706 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5973e076-24f8-4a72-a154-c5d4d2399805 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:45:51.485989 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5973e076-24f8-4a72-a154-c5d4d2399805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:45:51.486256 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5973e076-24f8-4a72-a154-c5d4d2399805 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:45:51.486896 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5973e076-24f8-4a72-a154-c5d4d2399805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:45:51.487929 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: WARNING py.warnings [None req-5973e076-24f8-4a72-a154-c5d4d2399805 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 03 12:45:51.487929 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: warnings.warn(msg) Oct 03 12:45:51.487929 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: Oct 03 12:45:51.488811 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5973e076-24f8-4a72-a154-c5d4d2399805 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:45:51.505195 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 764/1500] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:45:51 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:45:51.507374 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deac7c75-3bb3-4270-a951-8ceef2acc70e admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:45:51.508887 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-deac7c75-3bb3-4270-a951-8ceef2acc70e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:45:51.541443 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deac7c75-3bb3-4270-a951-8ceef2acc70e admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:45:51.634206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e5b93-ce23-4f77-b23a-e00f07c2ec0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:45:51.634786 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e5b93-ce23-4f77-b23a-e00f07c2ec0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:45:51.635061 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e5b93-ce23-4f77-b23a-e00f07c2ec0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:45:51.635355 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e5b93-ce23-4f77-b23a-e00f07c2ec0a admin admin] PATH_INFO: `/v3/projects/cc61fc81357d4056831ed333d6e80778/users/bd661df8f8634d2f9a7333f94cbf4767/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:45:51.657287 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5e5b93-ce23-4f77-b23a-e00f07c2ec0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc61fc81357d4056831ed333d6e80778, user_id=bd661df8f8634d2f9a7333f94cbf4767, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:45:51.658302 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: WARNING py.warnings [None req-3d5e5b93-ce23-4f77-b23a-e00f07c2ec0a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 12:45:51.658302 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: warnings.warn(msg) Oct 03 12:45:51.658302 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: Oct 03 12:45:51.660274 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5e5b93-ce23-4f77-b23a-e00f07c2ec0a admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:45:51.788862 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 737/1501] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:45:51 2022] PUT /identity/v3/projects/cc61fc81357d4056831ed333d6e80778/users/bd661df8f8634d2f9a7333f94cbf4767/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:45:51.801444 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55a612-22de-49c5-8e8f-4f644decdfe6 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:45:51.801725 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55a612-22de-49c5-8e8f-4f644decdfe6 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:45:51.801995 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55a612-22de-49c5-8e8f-4f644decdfe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:45:51.859174 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-cf55a612-22de-49c5-8e8f-4f644decdfe6 None None] MFA Rules not processed for user `bd661df8f8634d2f9a7333f94cbf4767`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:45:51.860391 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-cf55a612-22de-49c5-8e8f-4f644decdfe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:45:51.937250 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 765/1502] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:45:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:45:51.951308 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f61c9-919a-4b94-ae03-adcd7b23ca83 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:45:51.951625 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f61c9-919a-4b94-ae03-adcd7b23ca83 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:45:51.951932 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f61c9-919a-4b94-ae03-adcd7b23ca83 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:45:51.953296 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 738/1503] 10.222.0.33 () {54 vars in 963 bytes} [Mon Oct 3 12:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:45:51.957965 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feffc20e-179e-4326-8396-1556ab24ee25 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:45:51.958253 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feffc20e-179e-4326-8396-1556ab24ee25 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:45:51.958527 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feffc20e-179e-4326-8396-1556ab24ee25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:45:51.989982 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:45:52.035453 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-feffc20e-179e-4326-8396-1556ab24ee25 None None] MFA Rules not processed for user `524e9184eee1486cb071cad5c67050c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:45:52.036660 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-feffc20e-179e-4326-8396-1556ab24ee25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:45:52.111067 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 766/1504] 10.222.0.33 () {58 vars in 1060 bytes} [Mon Oct 3 12:45:51 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:45:52.119515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4f61c9-919a-4b94-ae03-adcd7b23ca83 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:45:52.121009 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4a4f61c9-919a-4b94-ae03-adcd7b23ca83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:45:52.169912 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:45:52.224235 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4f61c9-919a-4b94-ae03-adcd7b23ca83 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:45:52.301199 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2953a382-25ae-41c5-aa02-cdc8adab738d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:45:52.301804 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953a382-25ae-41c5-aa02-cdc8adab738d service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:45:52.302079 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953a382-25ae-41c5-aa02-cdc8adab738d service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:45:52.302348 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953a382-25ae-41c5-aa02-cdc8adab738d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:45:52.304567 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-2953a382-25ae-41c5-aa02-cdc8adab738d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:45:52.405847 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2953a382-25ae-41c5-aa02-cdc8adab738d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:45:52.407288 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2953a382-25ae-41c5-aa02-cdc8adab738d service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:45:52.410812 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:45:52.425418 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 739/1505] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:45:52 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:45:52.481946 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ed127-0575-49c4-b2b0-b760179eec6b None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:45:52.482368 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ed127-0575-49c4-b2b0-b760179eec6b None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:45:52.482524 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ed127-0575-49c4-b2b0-b760179eec6b None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:45:52.483823 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 767/1506] 10.222.0.33 () {52 vars in 830 bytes} [Mon Oct 3 12:45:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:45:52.487459 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2953a382-25ae-41c5-aa02-cdc8adab738d service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:45:52.488650 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-2953a382-25ae-41c5-aa02-cdc8adab738d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:45:52.587674 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2953a382-25ae-41c5-aa02-cdc8adab738d service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:45:52.681775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa4329-6792-497a-96dc-baf4a6556024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:45:52.682411 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa4329-6792-497a-96dc-baf4a6556024 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:45:52.686166 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa4329-6792-497a-96dc-baf4a6556024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:45:52.686166 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa4329-6792-497a-96dc-baf4a6556024 admin admin] PATH_INFO: `/v3/users/bd661df8f8634d2f9a7333f94cbf4767` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:45:52.696192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaa4329-6792-497a-96dc-baf4a6556024 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd661df8f8634d2f9a7333f94cbf4767)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:45:52.697207 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: WARNING py.warnings [None req-deaa4329-6792-497a-96dc-baf4a6556024 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 12:45:52.697207 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: warnings.warn(msg) Oct 03 12:45:52.697207 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: Oct 03 12:45:52.698455 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaa4329-6792-497a-96dc-baf4a6556024 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:45:52.708558 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:45:52.916796 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-deaa4329-6792-497a-96dc-baf4a6556024 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd661df8f8634d2f9a7333f94cbf4767'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:45:52.939409 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-deaa4329-6792-497a-96dc-baf4a6556024 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd661df8f8634d2f9a7333f94cbf4767'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:45:52.939877 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-deaa4329-6792-497a-96dc-baf4a6556024 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd661df8f8634d2f9a7333f94cbf4767'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:45:52.956522 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-deaa4329-6792-497a-96dc-baf4a6556024 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd661df8f8634d2f9a7333f94cbf4767'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:45:53.033433 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-deaa4329-6792-497a-96dc-baf4a6556024 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd661df8f8634d2f9a7333f94cbf4767'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:45:53.042759 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 740/1507] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:45:52 2022] DELETE /identity/v3/users/bd661df8f8634d2f9a7333f94cbf4767 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:45:53.047385 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c91c5-8601-4fc3-aac8-cfd9b92f148c None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:45:53.047672 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c91c5-8601-4fc3-aac8-cfd9b92f148c None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:45:53.047948 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c91c5-8601-4fc3-aac8-cfd9b92f148c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:45:53.105914 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-034c91c5-8601-4fc3-aac8-cfd9b92f148c None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:45:53.107154 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-034c91c5-8601-4fc3-aac8-cfd9b92f148c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:45:53.184897 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:45:53.197831 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 768/1508] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:45:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:45:53.207637 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa4329-6792-497a-96dc-baf4a6556024 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:45:53.208844 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-deaa4329-6792-497a-96dc-baf4a6556024 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:45:53.309225 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa4329-6792-497a-96dc-baf4a6556024 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:45:53.368463 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:45:53.387565 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3855c3-f640-4a72-8d27-fcd8e67054ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:45:53.388182 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3855c3-f640-4a72-8d27-fcd8e67054ba service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:45:53.388458 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3855c3-f640-4a72-8d27-fcd8e67054ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:45:53.388726 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3855c3-f640-4a72-8d27-fcd8e67054ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:45:53.390305 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-6c3855c3-f640-4a72-8d27-fcd8e67054ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:45:53.499149 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3855c3-f640-4a72-8d27-fcd8e67054ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:45:53.500665 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3855c3-f640-4a72-8d27-fcd8e67054ba service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:45:53.519239 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 741/1509] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:45:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:45:54.027958 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c91c5-8601-4fc3-aac8-cfd9b92f148c None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:45:54.029188 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-034c91c5-8601-4fc3-aac8-cfd9b92f148c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:45:54.041289 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c91c5-8601-4fc3-aac8-cfd9b92f148c None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:45:54.146535 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8d538-f8d0-4c24-bf1c-9caf23607295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:45:54.147862 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8d538-f8d0-4c24-bf1c-9caf23607295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:45:54.148206 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8d538-f8d0-4c24-bf1c-9caf23607295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:45:54.148480 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8d538-f8d0-4c24-bf1c-9caf23607295 admin admin] PATH_INFO: `/v3/projects/cc61fc81357d4056831ed333d6e80778` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:45:54.160989 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a8d538-f8d0-4c24-bf1c-9caf23607295 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc61fc81357d4056831ed333d6e80778)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:45:54.162040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: WARNING py.warnings [None req-c8a8d538-f8d0-4c24-bf1c-9caf23607295 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 12:45:54.162040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: warnings.warn(msg) Oct 03 12:45:54.162040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: Oct 03 12:45:54.163282 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a8d538-f8d0-4c24-bf1c-9caf23607295 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:45:54.294985 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-c8a8d538-f8d0-4c24-bf1c-9caf23607295 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc61fc81357d4056831ed333d6e80778', 'request_id': 'req-c8a8d538-f8d0-4c24-bf1c-9caf23607295'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:45:54.308976 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-c8a8d538-f8d0-4c24-bf1c-9caf23607295 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc61fc81357d4056831ed333d6e80778', 'request_id': 'req-c8a8d538-f8d0-4c24-bf1c-9caf23607295'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:45:54.376388 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-c8a8d538-f8d0-4c24-bf1c-9caf23607295 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc61fc81357d4056831ed333d6e80778', 'request_id': 'req-c8a8d538-f8d0-4c24-bf1c-9caf23607295'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:45:54.403836 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-c8a8d538-f8d0-4c24-bf1c-9caf23607295 admin admin] The token cache is being invalidate because project cc61fc81357d4056831ed333d6e80778 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:45:54.404130 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-c8a8d538-f8d0-4c24-bf1c-9caf23607295 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:45:54.404661 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-c8a8d538-f8d0-4c24-bf1c-9caf23607295 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:45:54.405958 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 769/1510] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:45:54 2022] DELETE /identity/v3/projects/cc61fc81357d4056831ed333d6e80778 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:46:03.517073 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=62 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:46:04.401861 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:47:26.653524 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15822a-7148-4270-a1fc-85f8ef623654 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:47:26.654129 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15822a-7148-4270-a1fc-85f8ef623654 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:47:26.654129 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15822a-7148-4270-a1fc-85f8ef623654 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:47:26.655345 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 742/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 12:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:47:26.660288 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a65e35-88e4-4fcb-8ec5-b04ba4ecbd12 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:47:26.660730 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a65e35-88e4-4fcb-8ec5-b04ba4ecbd12 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:47:26.660847 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a65e35-88e4-4fcb-8ec5-b04ba4ecbd12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:47:26.742252 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-09a65e35-88e4-4fcb-8ec5-b04ba4ecbd12 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:47:26.743495 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-09a65e35-88e4-4fcb-8ec5-b04ba4ecbd12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:47:26.848870 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 770/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 12:47:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:47:27.001920 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e15822a-7148-4270-a1fc-85f8ef623654 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:47:27.005539 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1e15822a-7148-4270-a1fc-85f8ef623654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:47:27.123171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e15822a-7148-4270-a1fc-85f8ef623654 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:47:27.128609 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e853ff-90ff-4576-ba82-db4d3f85409f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:47:27.129217 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e853ff-90ff-4576-ba82-db4d3f85409f service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:47:27.129496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e853ff-90ff-4576-ba82-db4d3f85409f service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:47:27.129769 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e853ff-90ff-4576-ba82-db4d3f85409f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:47:27.138478 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e853ff-90ff-4576-ba82-db4d3f85409f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:47:27.140091 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e853ff-90ff-4576-ba82-db4d3f85409f service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:47:27.149558 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 743/1513] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:47:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:47:28.246296 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b83b2-ec08-41f9-baf5-e4b62a01cb69 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:47:28.246915 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b83b2-ec08-41f9-baf5-e4b62a01cb69 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:47:28.246915 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b83b2-ec08-41f9-baf5-e4b62a01cb69 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:47:28.248256 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 771/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 12:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:47:28.255142 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c6f39f-8e90-49c9-8db2-b6921ef05491 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:47:28.255564 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c6f39f-8e90-49c9-8db2-b6921ef05491 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:47:28.255735 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c6f39f-8e90-49c9-8db2-b6921ef05491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:47:28.282088 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-e9c6f39f-8e90-49c9-8db2-b6921ef05491 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:47:28.282088 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e9c6f39f-8e90-49c9-8db2-b6921ef05491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:47:28.287130 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 744/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 12:47:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:47:28.442883 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b83b2-ec08-41f9-baf5-e4b62a01cb69 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:47:28.457607 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b83b2-ec08-41f9-baf5-e4b62a01cb69 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:47:28.462993 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7932ff1-a2f7-4293-a69e-745cf700ef24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:47:28.464123 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7932ff1-a2f7-4293-a69e-745cf700ef24 service nova] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:47:28.464404 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7932ff1-a2f7-4293-a69e-745cf700ef24 service nova] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:47:28.464675 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7932ff1-a2f7-4293-a69e-745cf700ef24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:47:28.472628 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7932ff1-a2f7-4293-a69e-745cf700ef24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:47:28.474192 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7932ff1-a2f7-4293-a69e-745cf700ef24 service nova] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:47:28.484530 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 772/1516] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:47:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:47:28.666553 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a195c3-b22e-47bd-8558-ed26cb640daf None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:47:28.666842 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a195c3-b22e-47bd-8558-ed26cb640daf None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:47:28.667116 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a195c3-b22e-47bd-8558-ed26cb640daf None None] PATH_INFO: `/v3` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:47:28.668446 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 745/1517] 10.222.0.46 () {54 vars in 925 bytes} [Mon Oct 3 12:47:28 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:47:28.675452 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ddfa42-fe93-4060-adf8-07ae1408f81c None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:47:28.675747 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ddfa42-fe93-4060-adf8-07ae1408f81c None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:47:28.676060 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ddfa42-fe93-4060-adf8-07ae1408f81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:47:28.738149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-56ddfa42-fe93-4060-adf8-07ae1408f81c None None] MFA Rules not processed for user `25eef64e92e9422ea62508c3eb0a197e`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:47:28.739360 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-56ddfa42-fe93-4060-adf8-07ae1408f81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:47:28.755951 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:47:28.809504 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 773/1518] 10.222.0.46 () {58 vars in 1015 bytes} [Mon Oct 3 12:47:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:47:32.611251 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4ac19-4d3b-4f1b-8f8c-86e39bd57a22 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:47:32.618523 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4ac19-4d3b-4f1b-8f8c-86e39bd57a22 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:47:32.618523 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4ac19-4d3b-4f1b-8f8c-86e39bd57a22 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:47:32.618523 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 746/1519] 10.222.0.33 () {54 vars in 961 bytes} [Mon Oct 3 12:47:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:47:32.618829 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b28a4-4cc8-45a8-bffe-4dae2a14764d None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:47:32.618829 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b28a4-4cc8-45a8-bffe-4dae2a14764d None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:47:32.618829 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b28a4-4cc8-45a8-bffe-4dae2a14764d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:47:32.638903 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-bc0b28a4-4cc8-45a8-bffe-4dae2a14764d None None] MFA Rules not processed for user `25eef64e92e9422ea62508c3eb0a197e`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:47:32.640119 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-bc0b28a4-4cc8-45a8-bffe-4dae2a14764d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:47:32.648939 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 774/1520] 10.222.0.33 () {58 vars in 1058 bytes} [Mon Oct 3 12:47:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:47:32.657356 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb4ac19-4d3b-4f1b-8f8c-86e39bd57a22 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:47:32.671057 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb4ac19-4d3b-4f1b-8f8c-86e39bd57a22 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:47:32.676470 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c42988-eb40-4f7e-93fa-1ebdefe217ec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25eef64e92e9422ea62508c3eb0a197e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:47:32.677061 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c42988-eb40-4f7e-93fa-1ebdefe217ec service placement] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:47:32.677391 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c42988-eb40-4f7e-93fa-1ebdefe217ec service placement] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:47:32.677686 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c42988-eb40-4f7e-93fa-1ebdefe217ec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:47:32.686372 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c42988-eb40-4f7e-93fa-1ebdefe217ec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:47:32.687993 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c42988-eb40-4f7e-93fa-1ebdefe217ec service placement] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:47:32.697615 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 747/1521] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:47:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:47:39.991478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540f2b1-8ec9-4419-b543-5204bf4d6fcf None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:47:39.992060 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540f2b1-8ec9-4419-b543-5204bf4d6fcf None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:47:39.992060 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540f2b1-8ec9-4419-b543-5204bf4d6fcf None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:47:39.993386 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 775/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 12:47:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:47:39.999722 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dca069-9d5e-4e72-b59e-27728b5a0210 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:47:40.000171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dca069-9d5e-4e72-b59e-27728b5a0210 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:47:40.000291 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dca069-9d5e-4e72-b59e-27728b5a0210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:47:40.021306 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-89dca069-9d5e-4e72-b59e-27728b5a0210 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:47:40.022485 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-89dca069-9d5e-4e72-b59e-27728b5a0210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:47:40.030955 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 748/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 12:47:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:47:40.182990 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540f2b1-8ec9-4419-b543-5204bf4d6fcf None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:47:40.197267 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540f2b1-8ec9-4419-b543-5204bf4d6fcf None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:47:40.203837 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c329f-ee74-4d0b-a5ac-575d1cbf6888 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:47:40.204431 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c329f-ee74-4d0b-a5ac-575d1cbf6888 service nova] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:47:40.204710 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c329f-ee74-4d0b-a5ac-575d1cbf6888 service nova] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:47:40.204982 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c329f-ee74-4d0b-a5ac-575d1cbf6888 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:47:40.212985 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360c329f-ee74-4d0b-a5ac-575d1cbf6888 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:47:40.215347 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360c329f-ee74-4d0b-a5ac-575d1cbf6888 service nova] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:47:40.224701 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 776/1524] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:47:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:47:41.335121 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b39de-78c3-428b-bde8-625d39303e8f None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:47:41.335696 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b39de-78c3-428b-bde8-625d39303e8f None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:47:41.335696 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b39de-78c3-428b-bde8-625d39303e8f None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:47:41.337041 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 749/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 12:47:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:47:41.343253 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff14d4-c026-47ab-b679-3257d1147bcc None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:47:41.343704 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff14d4-c026-47ab-b679-3257d1147bcc None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:47:41.343808 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff14d4-c026-47ab-b679-3257d1147bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:47:41.364173 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-c2ff14d4-c026-47ab-b679-3257d1147bcc None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:47:41.365361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-c2ff14d4-c026-47ab-b679-3257d1147bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:47:41.374111 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 777/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 12:47:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:47:41.526291 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b39de-78c3-428b-bde8-625d39303e8f None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:47:41.540878 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b39de-78c3-428b-bde8-625d39303e8f None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:47:41.549399 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a037270e-143a-48c8-a1b3-136a1fd20b6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:47:41.550379 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037270e-143a-48c8-a1b3-136a1fd20b6a service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:47:41.550890 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037270e-143a-48c8-a1b3-136a1fd20b6a service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:47:41.551425 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037270e-143a-48c8-a1b3-136a1fd20b6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:47:41.565560 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a037270e-143a-48c8-a1b3-136a1fd20b6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:47:41.567143 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a037270e-143a-48c8-a1b3-136a1fd20b6a service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:47:41.579347 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 750/1527] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:47:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:47:45.798491 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea11134-be3a-49b9-a882-c2c73951abe5 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:47:45.799069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea11134-be3a-49b9-a882-c2c73951abe5 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:47:45.799069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea11134-be3a-49b9-a882-c2c73951abe5 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:47:45.800420 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 778/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 12:47:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:47:45.805538 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7023bd9-9727-44e2-9c5e-edc53bce1616 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:47:45.805963 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7023bd9-9727-44e2-9c5e-edc53bce1616 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:47:45.806114 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7023bd9-9727-44e2-9c5e-edc53bce1616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:47:45.827392 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-d7023bd9-9727-44e2-9c5e-edc53bce1616 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:47:45.828607 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d7023bd9-9727-44e2-9c5e-edc53bce1616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:47:45.836487 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 751/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 12:47:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:47:46.132882 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea11134-be3a-49b9-a882-c2c73951abe5 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:47:46.134598 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-7ea11134-be3a-49b9-a882-c2c73951abe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:47:46.211946 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea11134-be3a-49b9-a882-c2c73951abe5 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:47:46.216313 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a6425-5d07-44d4-9ed0-e08591625e59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:47:46.216924 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a6425-5d07-44d4-9ed0-e08591625e59 service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:47:46.217207 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a6425-5d07-44d4-9ed0-e08591625e59 service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:47:46.217482 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a6425-5d07-44d4-9ed0-e08591625e59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:47:46.226254 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a6425-5d07-44d4-9ed0-e08591625e59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:47:46.227742 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a6425-5d07-44d4-9ed0-e08591625e59 service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:47:46.237626 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 779/1530] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:47:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:47:48.456019 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6dad1-8706-406d-a999-c33fc7246311 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:47:48.456641 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6dad1-8706-406d-a999-c33fc7246311 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:47:48.456641 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6dad1-8706-406d-a999-c33fc7246311 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:47:48.457906 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 752/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 12:47:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:47:48.463076 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d282132-d6d6-455c-a04f-a49258f8a25c None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:47:48.463561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d282132-d6d6-455c-a04f-a49258f8a25c None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:47:48.464214 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d282132-d6d6-455c-a04f-a49258f8a25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:47:48.484714 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-3d282132-d6d6-455c-a04f-a49258f8a25c None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:47:48.485890 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-3d282132-d6d6-455c-a04f-a49258f8a25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:47:48.494046 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 780/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 12:47:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:47:48.783886 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6dad1-8706-406d-a999-c33fc7246311 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:47:48.794968 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6dad1-8706-406d-a999-c33fc7246311 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:47:48.798409 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7133eb88-651f-4d38-bed9-2fa805883af9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:47:48.798997 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7133eb88-651f-4d38-bed9-2fa805883af9 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:47:48.799271 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7133eb88-651f-4d38-bed9-2fa805883af9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:47:48.799567 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7133eb88-651f-4d38-bed9-2fa805883af9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:47:48.808119 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7133eb88-651f-4d38-bed9-2fa805883af9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:47:48.809528 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7133eb88-651f-4d38-bed9-2fa805883af9 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:47:48.818641 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 753/1533] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:47:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:47:52.617278 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9685d-94bb-404a-a37c-7141c96d78ad None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:47:52.617851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9685d-94bb-404a-a37c-7141c96d78ad None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:47:52.617851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9685d-94bb-404a-a37c-7141c96d78ad None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:47:52.619076 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 781/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 12:47:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:47:52.624789 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e79b4-9097-4cd9-b015-1948b034c349 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:47:52.625239 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e79b4-9097-4cd9-b015-1948b034c349 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:47:52.625338 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e79b4-9097-4cd9-b015-1948b034c349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:47:52.646931 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-1e1e79b4-9097-4cd9-b015-1948b034c349 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:47:52.648143 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1e1e79b4-9097-4cd9-b015-1948b034c349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:47:52.655944 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 754/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 12:47:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:47:52.976959 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a9685d-94bb-404a-a37c-7141c96d78ad None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:47:52.978660 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-98a9685d-94bb-404a-a37c-7141c96d78ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:47:53.044376 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:47:53.058320 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a9685d-94bb-404a-a37c-7141c96d78ad None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:47:53.062353 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd43999e-3f43-4b6a-9288-cc57e2c4990c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:47:53.062559 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43999e-3f43-4b6a-9288-cc57e2c4990c service cinder] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:47:53.062838 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43999e-3f43-4b6a-9288-cc57e2c4990c service cinder] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:47:53.063109 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43999e-3f43-4b6a-9288-cc57e2c4990c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:47:53.071924 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd43999e-3f43-4b6a-9288-cc57e2c4990c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:47:53.073949 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd43999e-3f43-4b6a-9288-cc57e2c4990c service cinder] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:47:53.083546 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 782/1536] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:47:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:47:55.225576 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a077c6-4de0-432a-ac67-113c82c5644b None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:47:55.231969 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a077c6-4de0-432a-ac67-113c82c5644b None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:47:55.231969 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a077c6-4de0-432a-ac67-113c82c5644b None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:47:55.231969 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 755/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 12:47:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:47:55.233270 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210c42d-26bf-41f6-bade-623b84f4b310 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:47:55.233685 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210c42d-26bf-41f6-bade-623b84f4b310 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:47:55.233825 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210c42d-26bf-41f6-bade-623b84f4b310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:47:55.253944 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-a210c42d-26bf-41f6-bade-623b84f4b310 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:47:55.255127 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-a210c42d-26bf-41f6-bade-623b84f4b310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:47:55.263506 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 783/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 12:47:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:47:55.562761 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a077c6-4de0-432a-ac67-113c82c5644b None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:47:55.564472 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-c3a077c6-4de0-432a-ac67-113c82c5644b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:47:55.571908 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=84 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:47:55.578404 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a077c6-4de0-432a-ac67-113c82c5644b None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:47:55.582048 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580586bd-33b4-4621-b933-8ca2bfdbb8b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:47:55.582648 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580586bd-33b4-4621-b933-8ca2bfdbb8b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:47:55.582926 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580586bd-33b4-4621-b933-8ca2bfdbb8b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:47:55.583199 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580586bd-33b4-4621-b933-8ca2bfdbb8b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:47:55.592200 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580586bd-33b4-4621-b933-8ca2bfdbb8b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:47:55.593649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580586bd-33b4-4621-b933-8ca2bfdbb8b5 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:47:55.602902 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 756/1539] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:47:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:48:05.249044 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=11 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:48:05.606947 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=5 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:21.422732 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9489a908-a960-4900-abef-73dd62192993 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:21.433007 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9489a908-a960-4900-abef-73dd62192993 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:21.433007 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9489a908-a960-4900-abef-73dd62192993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:21.452877 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-9489a908-a960-4900-abef-73dd62192993 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:21.454087 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-9489a908-a960-4900-abef-73dd62192993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:21.466346 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 784/1540] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:49:21.470145 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf3c98-effb-4c37-9b03-625f459c391b None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:21.471655 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf3c98-effb-4c37-9b03-625f459c391b None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:21.471655 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf3c98-effb-4c37-9b03-625f459c391b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:21.492172 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad56c9f0-9286-4401-8d7b-1a86d67a42f3 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:21.492510 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad56c9f0-9286-4401-8d7b-1a86d67a42f3 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:21.492727 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad56c9f0-9286-4401-8d7b-1a86d67a42f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:21.512568 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-4ebf3c98-effb-4c37-9b03-625f459c391b None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:21.513706 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4ebf3c98-effb-4c37-9b03-625f459c391b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:21.519239 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-ad56c9f0-9286-4401-8d7b-1a86d67a42f3 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:21.520476 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-ad56c9f0-9286-4401-8d7b-1a86d67a42f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:21.526190 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 757/1541] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:21.536012 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6971200-0614-4eb1-8d8a-6a376000aaf1 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:21.536214 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 785/1542] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Oct 03 12:49:21.536619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6971200-0614-4eb1-8d8a-6a376000aaf1 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:21.536998 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6971200-0614-4eb1-8d8a-6a376000aaf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:21.540779 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad56c9f0-9286-4401-8d7b-1a86d67a42f3 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:21.552986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad56c9f0-9286-4401-8d7b-1a86d67a42f3 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:21.566436 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a98864e-c569-42e8-a2c3-45c3957464a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:21.567074 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98864e-c569-42e8-a2c3-45c3957464a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:21.567380 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98864e-c569-42e8-a2c3-45c3957464a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:21.567661 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98864e-c569-42e8-a2c3-45c3957464a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:21.568316 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a98864e-c569-42e8-a2c3-45c3957464a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:49:21.568620 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a98864e-c569-42e8-a2c3-45c3957464a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:21.570234 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a98864e-c569-42e8-a2c3-45c3957464a8 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:21.573790 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-b6971200-0614-4eb1-8d8a-6a376000aaf1 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:21.575033 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b6971200-0614-4eb1-8d8a-6a376000aaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:21.594837 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 758/1543] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:49:21.600258 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 786/1544] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:49:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:49:21.600456 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6971200-0614-4eb1-8d8a-6a376000aaf1 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:21.608182 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a98864e-c569-42e8-a2c3-45c3957464a8 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:21.616337 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6971200-0614-4eb1-8d8a-6a376000aaf1 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:21.624381 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306af5a-815c-4127-bf13-a55b08a35ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:21.625020 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306af5a-815c-4127-bf13-a55b08a35ecf admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:21.625293 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306af5a-815c-4127-bf13-a55b08a35ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:21.625560 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306af5a-815c-4127-bf13-a55b08a35ecf admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:21.626355 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2306af5a-815c-4127-bf13-a55b08a35ecf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:49:21.627195 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2306af5a-815c-4127-bf13-a55b08a35ecf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:21.628264 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: WARNING py.warnings [None req-2306af5a-815c-4127-bf13-a55b08a35ecf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 03 12:49:21.628264 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: warnings.warn(msg) Oct 03 12:49:21.628264 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: Oct 03 12:49:21.629161 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2306af5a-815c-4127-bf13-a55b08a35ecf admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:21.629292 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a98864e-c569-42e8-a2c3-45c3957464a8 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:21.644110 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25441577-0da9-42e5-9bd0-0b5bd6587c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:21.644752 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25441577-0da9-42e5-9bd0-0b5bd6587c47 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:21.645039 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25441577-0da9-42e5-9bd0-0b5bd6587c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:21.645318 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25441577-0da9-42e5-9bd0-0b5bd6587c47 admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:21.646198 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25441577-0da9-42e5-9bd0-0b5bd6587c47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:21.647249 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: WARNING py.warnings [None req-25441577-0da9-42e5-9bd0-0b5bd6587c47 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 12:49:21.647249 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: warnings.warn(msg) Oct 03 12:49:21.647249 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: Oct 03 12:49:21.648794 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25441577-0da9-42e5-9bd0-0b5bd6587c47 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:21.655488 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 759/1545] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:49:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:49:21.656808 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee10e7b-bede-460e-8583-3ad8d9a08609 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:21.657092 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee10e7b-bede-460e-8583-3ad8d9a08609 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:21.657365 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee10e7b-bede-460e-8583-3ad8d9a08609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:21.692344 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-6ee10e7b-bede-460e-8583-3ad8d9a08609 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:21.693559 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-6ee10e7b-bede-460e-8583-3ad8d9a08609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:21.718459 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 760/1546] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:21.719918 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39e0cb-615a-45e9-9a22-c82cb1bc9e2c None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:21.720263 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39e0cb-615a-45e9-9a22-c82cb1bc9e2c None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:21.720514 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39e0cb-615a-45e9-9a22-c82cb1bc9e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:21.744877 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-ef39e0cb-615a-45e9-9a22-c82cb1bc9e2c None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:21.746083 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ef39e0cb-615a-45e9-9a22-c82cb1bc9e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:21.755567 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 761/1547] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:21.756489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef39e0cb-615a-45e9-9a22-c82cb1bc9e2c None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:21.768394 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef39e0cb-615a-45e9-9a22-c82cb1bc9e2c None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:21.773238 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383debe8-8669-4e10-bd63-bbb93010f143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:21.773856 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383debe8-8669-4e10-bd63-bbb93010f143 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:21.774132 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383debe8-8669-4e10-bd63-bbb93010f143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:21.774404 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383debe8-8669-4e10-bd63-bbb93010f143 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:21.775028 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383debe8-8669-4e10-bd63-bbb93010f143 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:21.777018 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383debe8-8669-4e10-bd63-bbb93010f143 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:21.795912 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 787/1548] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:49:21 2022] POST /identity/v3/projects => generated 393 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:49:21.797980 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf097b3-f418-48c2-8bf0-d5d5998068c2 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:21.799535 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf097b3-f418-48c2-8bf0-d5d5998068c2 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:21.805883 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf097b3-f418-48c2-8bf0-d5d5998068c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:21.822603 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-bdf097b3-f418-48c2-8bf0-d5d5998068c2 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:21.824474 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-bdf097b3-f418-48c2-8bf0-d5d5998068c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:21.893423 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 788/1549] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:21.894669 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2358649-e5e2-47a4-980e-536483a62676 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:21.894959 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2358649-e5e2-47a4-980e-536483a62676 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:21.895236 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2358649-e5e2-47a4-980e-536483a62676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:21.908966 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 762/1550] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:49:21 2022] POST /identity/v3/projects => generated 363 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:49:21.908966 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c142e48-ae6e-41af-9015-f240df8da9df None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:21.908966 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c142e48-ae6e-41af-9015-f240df8da9df None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:21.908966 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c142e48-ae6e-41af-9015-f240df8da9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:21.915606 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-d2358649-e5e2-47a4-980e-536483a62676 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:21.916912 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-d2358649-e5e2-47a4-980e-536483a62676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:21.928510 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-6c142e48-ae6e-41af-9015-f240df8da9df None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:21.929674 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-6c142e48-ae6e-41af-9015-f240df8da9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:21.976679 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 789/1551] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:21.977915 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e08d5-1c10-48bb-a9f3-6ef609b32300 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:21.978198 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e08d5-1c10-48bb-a9f3-6ef609b32300 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:21.978471 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e08d5-1c10-48bb-a9f3-6ef609b32300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:21.997297 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 763/1552] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:21.998110 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c142e48-ae6e-41af-9015-f240df8da9df None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:21.998254 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-188e08d5-1c10-48bb-a9f3-6ef609b32300 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:21.999372 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-188e08d5-1c10-48bb-a9f3-6ef609b32300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:22.008040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 790/1553] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:22.009273 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b16850-c10e-4331-9ae1-e51c27323061 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:22.009950 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b16850-c10e-4331-9ae1-e51c27323061 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:22.009950 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b16850-c10e-4331-9ae1-e51c27323061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:22.013707 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c142e48-ae6e-41af-9015-f240df8da9df None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:22.019342 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ebfb1-5249-4c4e-ac42-27d9a752233f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:22.019899 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ebfb1-5249-4c4e-ac42-27d9a752233f admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:22.020183 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ebfb1-5249-4c4e-ac42-27d9a752233f admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:22.020461 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ebfb1-5249-4c4e-ac42-27d9a752233f admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:22.021122 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922ebfb1-5249-4c4e-ac42-27d9a752233f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:22.022196 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: WARNING py.warnings [None req-922ebfb1-5249-4c4e-ac42-27d9a752233f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 12:49:22.022196 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: warnings.warn(msg) Oct 03 12:49:22.022196 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: Oct 03 12:49:22.023565 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922ebfb1-5249-4c4e-ac42-27d9a752233f admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:22.028673 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-c9b16850-c10e-4331-9ae1-e51c27323061 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:22.029957 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-c9b16850-c10e-4331-9ae1-e51c27323061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:22.037897 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 791/1554] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:22.040539 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d4f16-1f40-4244-b5af-586d60ec59ee None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:22.040821 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d4f16-1f40-4244-b5af-586d60ec59ee None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:22.041123 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d4f16-1f40-4244-b5af-586d60ec59ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:22.060825 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-373d4f16-1f40-4244-b5af-586d60ec59ee None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:22.061994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-373d4f16-1f40-4244-b5af-586d60ec59ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:22.070464 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 792/1555] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:22.071268 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373d4f16-1f40-4244-b5af-586d60ec59ee None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:22.080437 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373d4f16-1f40-4244-b5af-586d60ec59ee None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:22.085049 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9a85c-3e93-4207-b995-75149ba4abd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:22.085652 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9a85c-3e93-4207-b995-75149ba4abd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:22.085928 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9a85c-3e93-4207-b995-75149ba4abd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:22.086228 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9a85c-3e93-4207-b995-75149ba4abd6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:22.086846 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f9a85c-3e93-4207-b995-75149ba4abd6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:49:22.087144 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f9a85c-3e93-4207-b995-75149ba4abd6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:22.088751 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f9a85c-3e93-4207-b995-75149ba4abd6 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:22.100078 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 793/1556] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:49:22.100802 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9a85c-3e93-4207-b995-75149ba4abd6 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:22.109857 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9a85c-3e93-4207-b995-75149ba4abd6 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:22.115203 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094f5a7-216b-44d3-a120-ce7b16c94bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:22.115795 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094f5a7-216b-44d3-a120-ce7b16c94bd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:22.116100 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094f5a7-216b-44d3-a120-ce7b16c94bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:22.116368 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094f5a7-216b-44d3-a120-ce7b16c94bd9 admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:22.116971 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f094f5a7-216b-44d3-a120-ce7b16c94bd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:22.118536 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 764/1557] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:49:21 2022] POST /identity/v3/users => generated 297 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:49:22.118821 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f094f5a7-216b-44d3-a120-ce7b16c94bd9 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:22.119833 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11edcc5-72a6-4773-bdf6-4b53e25afa0a None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:22.125582 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11edcc5-72a6-4773-bdf6-4b53e25afa0a None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:22.125582 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11edcc5-72a6-4773-bdf6-4b53e25afa0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:22.130101 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:22.141601 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-d11edcc5-72a6-4773-bdf6-4b53e25afa0a None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:22.143356 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d11edcc5-72a6-4773-bdf6-4b53e25afa0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:22.151190 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 765/1558] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:22.152983 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff463c9a-12b2-4a83-8bc3-015122f2d67a None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:22.152983 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff463c9a-12b2-4a83-8bc3-015122f2d67a None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:22.153215 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff463c9a-12b2-4a83-8bc3-015122f2d67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:22.172104 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-ff463c9a-12b2-4a83-8bc3-015122f2d67a None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:22.175657 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ff463c9a-12b2-4a83-8bc3-015122f2d67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:22.182396 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 766/1559] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:22.183163 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff463c9a-12b2-4a83-8bc3-015122f2d67a None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:22.193472 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff463c9a-12b2-4a83-8bc3-015122f2d67a None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:22.198714 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd710ff-2a52-400f-b06e-46e6f5cfaa4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:22.198848 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd710ff-2a52-400f-b06e-46e6f5cfaa4d admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:22.199022 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd710ff-2a52-400f-b06e-46e6f5cfaa4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:22.199290 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd710ff-2a52-400f-b06e-46e6f5cfaa4d admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:22.199952 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd710ff-2a52-400f-b06e-46e6f5cfaa4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:49:22.200250 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd710ff-2a52-400f-b06e-46e6f5cfaa4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:22.201827 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd710ff-2a52-400f-b06e-46e6f5cfaa4d admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:22.214121 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 767/1560] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:49:22.215507 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c32e7-3ed9-4a50-8633-1a50b4760eea None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:22.215788 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c32e7-3ed9-4a50-8633-1a50b4760eea None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:22.216056 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c32e7-3ed9-4a50-8633-1a50b4760eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:22.226947 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 794/1561] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:49:22 2022] POST /identity/v3/users => generated 282 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:49:22.229208 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71aeac-ed83-443c-ba19-8a95f549c9b6 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:22.229208 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71aeac-ed83-443c-ba19-8a95f549c9b6 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:22.229208 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71aeac-ed83-443c-ba19-8a95f549c9b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:22.241985 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-785c32e7-3ed9-4a50-8633-1a50b4760eea None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:22.243180 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-785c32e7-3ed9-4a50-8633-1a50b4760eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:22.250055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-dd71aeac-ed83-443c-ba19-8a95f549c9b6 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:22.251246 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-dd71aeac-ed83-443c-ba19-8a95f549c9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:22.254940 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 768/1562] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:22.254940 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c32e7-3ed9-4a50-8633-1a50b4760eea None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:22.260722 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 795/1563] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:22.261522 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71aeac-ed83-443c-ba19-8a95f549c9b6 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:22.265074 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c32e7-3ed9-4a50-8633-1a50b4760eea None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:22.270024 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774d45dc-4a3e-4c02-8e0c-a8684ade4d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:22.270563 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d45dc-4a3e-4c02-8e0c-a8684ade4d0d admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:22.270840 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d45dc-4a3e-4c02-8e0c-a8684ade4d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:22.271109 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d45dc-4a3e-4c02-8e0c-a8684ade4d0d admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:22.271207 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71aeac-ed83-443c-ba19-8a95f549c9b6 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:22.271760 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774d45dc-4a3e-4c02-8e0c-a8684ade4d0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:22.273646 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774d45dc-4a3e-4c02-8e0c-a8684ade4d0d admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:22.275772 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cca3ab-a965-4966-85d8-f69fb92b553e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:22.276360 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cca3ab-a965-4966-85d8-f69fb92b553e admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:22.276641 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cca3ab-a965-4966-85d8-f69fb92b553e admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:22.276916 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cca3ab-a965-4966-85d8-f69fb92b553e admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:22.277551 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cca3ab-a965-4966-85d8-f69fb92b553e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:22.279154 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cca3ab-a965-4966-85d8-f69fb92b553e admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:22.289177 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 796/1564] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:49:22.289973 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cca3ab-a965-4966-85d8-f69fb92b553e admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:22.301338 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cca3ab-a965-4966-85d8-f69fb92b553e admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:22.305884 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8670720c-dbc1-479e-bbde-d8f1b8849c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:22.306465 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8670720c-dbc1-479e-bbde-d8f1b8849c10 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:22.306773 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8670720c-dbc1-479e-bbde-d8f1b8849c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:22.307045 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8670720c-dbc1-479e-bbde-d8f1b8849c10 admin admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:22.307719 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8670720c-dbc1-479e-bbde-d8f1b8849c10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:49:22.308037 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8670720c-dbc1-479e-bbde-d8f1b8849c10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:22.309579 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8670720c-dbc1-479e-bbde-d8f1b8849c10 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:22.321811 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 797/1565] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:49:22.322589 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8670720c-dbc1-479e-bbde-d8f1b8849c10 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:22.333968 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8670720c-dbc1-479e-bbde-d8f1b8849c10 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:22.338512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d2bfc-641b-4bd1-aeab-d4d4a8dd4686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:22.339075 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d2bfc-641b-4bd1-aeab-d4d4a8dd4686 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:22.339383 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d2bfc-641b-4bd1-aeab-d4d4a8dd4686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:22.339662 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d2bfc-641b-4bd1-aeab-d4d4a8dd4686 admin admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:22.340283 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6d2bfc-641b-4bd1-aeab-d4d4a8dd4686 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:49:22.340589 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6d2bfc-641b-4bd1-aeab-d4d4a8dd4686 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:22.342139 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6d2bfc-641b-4bd1-aeab-d4d4a8dd4686 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:22.353421 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 798/1566] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:49:22.358361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d2bfc-641b-4bd1-aeab-d4d4a8dd4686 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:22.358859 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 769/1567] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:49:22 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:49:22.359998 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774d45dc-4a3e-4c02-8e0c-a8684ade4d0d admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:22.367994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d2bfc-641b-4bd1-aeab-d4d4a8dd4686 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:22.392993 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:22.425645 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774d45dc-4a3e-4c02-8e0c-a8684ade4d0d admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:22.429826 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ba3b8-8fa7-4dc7-95aa-3af024ebbb0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:22.430439 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f7e03-b3df-4da9-81a1-a1286c6a6b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:22.435703 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ba3b8-8fa7-4dc7-95aa-3af024ebbb0e admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:22.435703 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ba3b8-8fa7-4dc7-95aa-3af024ebbb0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:22.435703 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ba3b8-8fa7-4dc7-95aa-3af024ebbb0e admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:22.436033 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f7e03-b3df-4da9-81a1-a1286c6a6b60 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:22.436033 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f7e03-b3df-4da9-81a1-a1286c6a6b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:22.436199 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421ba3b8-8fa7-4dc7-95aa-3af024ebbb0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:22.436302 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f7e03-b3df-4da9-81a1-a1286c6a6b60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:22.436302 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56f7e03-b3df-4da9-81a1-a1286c6a6b60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:22.436472 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421ba3b8-8fa7-4dc7-95aa-3af024ebbb0e admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:22.436559 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56f7e03-b3df-4da9-81a1-a1286c6a6b60 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:22.445778 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 770/1568] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:49:22.446745 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f7e03-b3df-4da9-81a1-a1286c6a6b60 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:22.456509 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f7e03-b3df-4da9-81a1-a1286c6a6b60 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:22.461162 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101001fc-c467-4227-b9fb-27adfe942832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:22.461740 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101001fc-c467-4227-b9fb-27adfe942832 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:22.462016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101001fc-c467-4227-b9fb-27adfe942832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:22.462286 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101001fc-c467-4227-b9fb-27adfe942832 admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:22.462909 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101001fc-c467-4227-b9fb-27adfe942832 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:49:22.463211 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101001fc-c467-4227-b9fb-27adfe942832 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:22.464853 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101001fc-c467-4227-b9fb-27adfe942832 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:22.476971 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 771/1569] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:49:22.481109 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101001fc-c467-4227-b9fb-27adfe942832 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:22.489003 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101001fc-c467-4227-b9fb-27adfe942832 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:22.493537 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e9edf4-eaea-4e15-a519-8a1a51edfddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:22.494104 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e9edf4-eaea-4e15-a519-8a1a51edfddb admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:22.494380 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e9edf4-eaea-4e15-a519-8a1a51edfddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:22.494648 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e9edf4-eaea-4e15-a519-8a1a51edfddb admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:22.495258 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e9edf4-eaea-4e15-a519-8a1a51edfddb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:49:22.495589 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e9edf4-eaea-4e15-a519-8a1a51edfddb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:22.497125 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e9edf4-eaea-4e15-a519-8a1a51edfddb admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:22.509482 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 772/1570] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:49:22.510113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e9edf4-eaea-4e15-a519-8a1a51edfddb admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:22.519253 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e9edf4-eaea-4e15-a519-8a1a51edfddb admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:22.523919 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec15f53-be0b-46ee-bd65-bc721d63827c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:22.524421 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec15f53-be0b-46ee-bd65-bc721d63827c admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:22.524696 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec15f53-be0b-46ee-bd65-bc721d63827c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:22.525013 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec15f53-be0b-46ee-bd65-bc721d63827c admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:22.525621 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec15f53-be0b-46ee-bd65-bc721d63827c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:22.527402 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec15f53-be0b-46ee-bd65-bc721d63827c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:22.537544 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 773/1571] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:49:22.538310 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec15f53-be0b-46ee-bd65-bc721d63827c admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:22.547602 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 799/1572] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:49:22 2022] POST /identity/v3/projects => generated 389 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:49:22.549051 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ba3b8-8fa7-4dc7-95aa-3af024ebbb0e admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:22.550070 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec15f53-be0b-46ee-bd65-bc721d63827c admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:22.607853 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51abeafa-ccd2-4951-9092-23eea00f3f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:22.608470 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51abeafa-ccd2-4951-9092-23eea00f3f76 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:22.608751 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51abeafa-ccd2-4951-9092-23eea00f3f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:22.609027 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51abeafa-ccd2-4951-9092-23eea00f3f76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:22.609655 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51abeafa-ccd2-4951-9092-23eea00f3f76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:22.612381 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ba3b8-8fa7-4dc7-95aa-3af024ebbb0e admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:22.613090 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51abeafa-ccd2-4951-9092-23eea00f3f76 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:22.616957 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e42f56-98b7-421e-a48b-60a1f22fab77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:22.617565 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e42f56-98b7-421e-a48b-60a1f22fab77 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:22.617846 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e42f56-98b7-421e-a48b-60a1f22fab77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:22.618124 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e42f56-98b7-421e-a48b-60a1f22fab77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:22.618752 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e42f56-98b7-421e-a48b-60a1f22fab77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:22.620681 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e42f56-98b7-421e-a48b-60a1f22fab77 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:22.697255 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 774/1573] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:49:22 2022] POST /identity/v3/projects => generated 377 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:49:22.697255 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51abeafa-ccd2-4951-9092-23eea00f3f76 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:22.707024 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 800/1574] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:49:22 2022] POST /identity/v3/projects => generated 381 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:49:22.707979 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e42f56-98b7-421e-a48b-60a1f22fab77 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:22.729935 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:22.760976 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51abeafa-ccd2-4951-9092-23eea00f3f76 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:22.772983 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e42f56-98b7-421e-a48b-60a1f22fab77 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:22.773780 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:22.780473 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95b2cfc-712b-43bb-ba64-2c19f55e5b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:22.780473 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b2cfc-712b-43bb-ba64-2c19f55e5b0e admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:22.780473 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b2cfc-712b-43bb-ba64-2c19f55e5b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:22.780473 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b2cfc-712b-43bb-ba64-2c19f55e5b0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:22.780944 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95b2cfc-712b-43bb-ba64-2c19f55e5b0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:22.781132 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95b2cfc-712b-43bb-ba64-2c19f55e5b0e admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:22.791752 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 801/1575] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:49:22.792585 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95b2cfc-712b-43bb-ba64-2c19f55e5b0e admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:22.802397 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95b2cfc-712b-43bb-ba64-2c19f55e5b0e admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:22.807161 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0196989-a4dd-4e57-9482-645228e6714f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:22.807779 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0196989-a4dd-4e57-9482-645228e6714f admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:22.808060 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0196989-a4dd-4e57-9482-645228e6714f admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:22.808332 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0196989-a4dd-4e57-9482-645228e6714f admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:22.808946 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0196989-a4dd-4e57-9482-645228e6714f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:22.810805 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0196989-a4dd-4e57-9482-645228e6714f admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:22.823916 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ce9c4-ae12-4351-8d69-d91dea63927d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:22.824542 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ce9c4-ae12-4351-8d69-d91dea63927d admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:22.824822 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ce9c4-ae12-4351-8d69-d91dea63927d admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:22.825097 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ce9c4-ae12-4351-8d69-d91dea63927d admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:22.825721 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050ce9c4-ae12-4351-8d69-d91dea63927d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:22.827676 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050ce9c4-ae12-4351-8d69-d91dea63927d admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:22.896144 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 802/1576] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:49:22 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:49:22.896992 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0196989-a4dd-4e57-9482-645228e6714f admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:22.912243 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 775/1577] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:49:22 2022] POST /identity/v3/users => generated 287 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:49:22.915343 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ce9c4-ae12-4351-8d69-d91dea63927d admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:22.923206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ce9c4-ae12-4351-8d69-d91dea63927d admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:22.962797 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0196989-a4dd-4e57-9482-645228e6714f admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:22.967515 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc09b46d-8a70-41dc-9226-c9830c720d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:22.968134 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc09b46d-8a70-41dc-9226-c9830c720d06 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:22.968418 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc09b46d-8a70-41dc-9226-c9830c720d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:22.968706 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc09b46d-8a70-41dc-9226-c9830c720d06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:22.969317 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc09b46d-8a70-41dc-9226-c9830c720d06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:22.972116 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc09b46d-8a70-41dc-9226-c9830c720d06 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:22.982807 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc4440-5c10-4753-903a-657321936cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:22.983498 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc4440-5c10-4753-903a-657321936cda admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:22.983777 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc4440-5c10-4753-903a-657321936cda admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:22.984047 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc4440-5c10-4753-903a-657321936cda admin admin] PATH_INFO: `/v3/projects/3f2b18d217d14d39886718cf209eb576/users/ad5654cfa65f4cfa9e68832b9cd3d129/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:22.998280 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc4440-5c10-4753-903a-657321936cda admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f2b18d217d14d39886718cf209eb576, user_id=ad5654cfa65f4cfa9e68832b9cd3d129, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:23.000964 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc4440-5c10-4753-903a-657321936cda admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:23.072640 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 803/1578] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:49:22 2022] POST /identity/v3/projects => generated 377 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:49:23.073486 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc09b46d-8a70-41dc-9226-c9830c720d06 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:23.105734 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 776/1579] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:49:22 2022] PUT /identity/v3/projects/3f2b18d217d14d39886718cf209eb576/users/ad5654cfa65f4cfa9e68832b9cd3d129/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:49:23.106602 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc4440-5c10-4753-903a-657321936cda admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:23.137254 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc09b46d-8a70-41dc-9226-c9830c720d06 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:23.162062 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:23.171625 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc4440-5c10-4753-903a-657321936cda admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:23.176694 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c034030c-e045-411f-bd3e-8f98b8d87a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:23.177309 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034030c-e045-411f-bd3e-8f98b8d87a3a admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:23.177586 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034030c-e045-411f-bd3e-8f98b8d87a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:23.177857 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034030c-e045-411f-bd3e-8f98b8d87a3a admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:23.178701 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c034030c-e045-411f-bd3e-8f98b8d87a3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:23.180667 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c034030c-e045-411f-bd3e-8f98b8d87a3a admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:23.195408 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d04f4de-74ca-47e3-b556-1c4c5349538e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:23.197811 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04f4de-74ca-47e3-b556-1c4c5349538e admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:23.197811 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04f4de-74ca-47e3-b556-1c4c5349538e admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:23.197811 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04f4de-74ca-47e3-b556-1c4c5349538e admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:23.197811 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d04f4de-74ca-47e3-b556-1c4c5349538e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:23.199214 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d04f4de-74ca-47e3-b556-1c4c5349538e admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:23.272701 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 777/1580] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:49:23 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:49:23.273734 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c034030c-e045-411f-bd3e-8f98b8d87a3a admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:23.283907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c034030c-e045-411f-bd3e-8f98b8d87a3a admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:23.288954 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f6a11-d6fc-4d3e-b7c8-08e7aa2c911b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:23.295800 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 804/1581] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:49:23 2022] POST /identity/v3/users => generated 295 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:49:23.295937 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f6a11-d6fc-4d3e-b7c8-08e7aa2c911b admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:23.296048 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d04f4de-74ca-47e3-b556-1c4c5349538e admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:23.296156 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f6a11-d6fc-4d3e-b7c8-08e7aa2c911b admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:23.296156 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f6a11-d6fc-4d3e-b7c8-08e7aa2c911b admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:23.296156 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461f6a11-d6fc-4d3e-b7c8-08e7aa2c911b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:23.296156 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461f6a11-d6fc-4d3e-b7c8-08e7aa2c911b admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:23.301953 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d04f4de-74ca-47e3-b556-1c4c5349538e admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:23.306892 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76798203-f3a4-410c-996f-b8deaebf0f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:23.307653 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76798203-f3a4-410c-996f-b8deaebf0f88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:23.307933 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76798203-f3a4-410c-996f-b8deaebf0f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:23.308207 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76798203-f3a4-410c-996f-b8deaebf0f88 admin admin] PATH_INFO: `/v3/projects/e999f3994f1a4d4c8bf59df277dfa2e6/users/461cbea0d41d405ab9be8af902268ee6/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:23.324310 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76798203-f3a4-410c-996f-b8deaebf0f88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e999f3994f1a4d4c8bf59df277dfa2e6, user_id=461cbea0d41d405ab9be8af902268ee6, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:23.324489 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: WARNING py.warnings [None req-76798203-f3a4-410c-996f-b8deaebf0f88 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 12:49:23.324489 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: warnings.warn(msg) Oct 03 12:49:23.324489 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: Oct 03 12:49:23.326349 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76798203-f3a4-410c-996f-b8deaebf0f88 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:23.379833 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 778/1582] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:49:23 2022] POST /identity/v3/users => generated 291 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:49:23.380831 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f6a11-d6fc-4d3e-b7c8-08e7aa2c911b admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:23.385927 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:23.391090 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f6a11-d6fc-4d3e-b7c8-08e7aa2c911b admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:23.396529 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2a3af5-bbf4-4b32-8fb5-7284646cdd74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:23.397114 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a3af5-bbf4-4b32-8fb5-7284646cdd74 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:23.397394 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a3af5-bbf4-4b32-8fb5-7284646cdd74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:23.397670 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a3af5-bbf4-4b32-8fb5-7284646cdd74 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:23.398283 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a3af5-bbf4-4b32-8fb5-7284646cdd74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:23.400290 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a3af5-bbf4-4b32-8fb5-7284646cdd74 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:23.408593 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 805/1583] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:49:23 2022] PUT /identity/v3/projects/e999f3994f1a4d4c8bf59df277dfa2e6/users/461cbea0d41d405ab9be8af902268ee6/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:49:23.409515 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76798203-f3a4-410c-996f-b8deaebf0f88 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:23.471384 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76798203-f3a4-410c-996f-b8deaebf0f88 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:23.476027 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f725d30-e2ce-4c20-972b-3ab6bd8055dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:23.476636 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f725d30-e2ce-4c20-972b-3ab6bd8055dd admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:23.476914 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f725d30-e2ce-4c20-972b-3ab6bd8055dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:23.477185 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f725d30-e2ce-4c20-972b-3ab6bd8055dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:23.477813 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f725d30-e2ce-4c20-972b-3ab6bd8055dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:23.479462 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f725d30-e2ce-4c20-972b-3ab6bd8055dd admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:23.490327 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 806/1584] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:49:23.496584 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f725d30-e2ce-4c20-972b-3ab6bd8055dd admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:23.500169 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f725d30-e2ce-4c20-972b-3ab6bd8055dd admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:23.504686 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda9a972-da36-42f1-8b67-47917ef3a1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:23.505243 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda9a972-da36-42f1-8b67-47917ef3a1f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:23.505521 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda9a972-da36-42f1-8b67-47917ef3a1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:23.505789 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda9a972-da36-42f1-8b67-47917ef3a1f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:23.506383 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda9a972-da36-42f1-8b67-47917ef3a1f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:23.508267 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda9a972-da36-42f1-8b67-47917ef3a1f5 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:23.521802 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 779/1585] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:49:23 2022] POST /identity/v3/users => generated 288 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:49:23.523050 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78132ea3-9e07-4028-98db-ba64de5a0381 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:23.525471 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78132ea3-9e07-4028-98db-ba64de5a0381 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:23.525471 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78132ea3-9e07-4028-98db-ba64de5a0381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:23.544145 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-78132ea3-9e07-4028-98db-ba64de5a0381 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:23.545647 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-78132ea3-9e07-4028-98db-ba64de5a0381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:23.553874 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 780/1586] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:23.554631 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78132ea3-9e07-4028-98db-ba64de5a0381 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:23.563813 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78132ea3-9e07-4028-98db-ba64de5a0381 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:23.568385 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8112bc-0b84-48bb-b544-f6e426abb708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:23.568974 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8112bc-0b84-48bb-b544-f6e426abb708 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:23.569253 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8112bc-0b84-48bb-b544-f6e426abb708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:23.569524 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8112bc-0b84-48bb-b544-f6e426abb708 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:23.570154 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8112bc-0b84-48bb-b544-f6e426abb708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:23.571809 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8112bc-0b84-48bb-b544-f6e426abb708 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:23.582137 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 781/1587] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:49:23.582577 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8112bc-0b84-48bb-b544-f6e426abb708 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:23.592009 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8112bc-0b84-48bb-b544-f6e426abb708 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:23.597594 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49697b2-3b41-4b80-8b25-80390a57aff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:23.598176 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49697b2-3b41-4b80-8b25-80390a57aff4 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:23.598447 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49697b2-3b41-4b80-8b25-80390a57aff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:23.598721 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49697b2-3b41-4b80-8b25-80390a57aff4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:23.599353 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49697b2-3b41-4b80-8b25-80390a57aff4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:23.600927 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49697b2-3b41-4b80-8b25-80390a57aff4 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:23.607275 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 807/1588] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:49:23 2022] POST /identity/v3/users => generated 289 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:49:23.608160 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda9a972-da36-42f1-8b67-47917ef3a1f5 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:23.617218 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 782/1589] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:49:23.619291 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda9a972-da36-42f1-8b67-47917ef3a1f5 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:23.621334 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994d5d6-f19e-4af0-8a07-bcd0e9beba1b None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:23.621614 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994d5d6-f19e-4af0-8a07-bcd0e9beba1b None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:23.621893 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994d5d6-f19e-4af0-8a07-bcd0e9beba1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:23.624335 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad5c856-6d70-466d-9889-bf6559fa6e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:23.624945 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5c856-6d70-466d-9889-bf6559fa6e4d admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:23.625228 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5c856-6d70-466d-9889-bf6559fa6e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:23.625500 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5c856-6d70-466d-9889-bf6559fa6e4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:23.626138 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad5c856-6d70-466d-9889-bf6559fa6e4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:23.628003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad5c856-6d70-466d-9889-bf6559fa6e4d admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:23.638996 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 808/1590] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:49:23.639866 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad5c856-6d70-466d-9889-bf6559fa6e4d admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:23.650876 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-8994d5d6-f19e-4af0-8a07-bcd0e9beba1b None None] MFA Rules not processed for user `461cbea0d41d405ab9be8af902268ee6`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:23.651627 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad5c856-6d70-466d-9889-bf6559fa6e4d admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:23.652121 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-8994d5d6-f19e-4af0-8a07-bcd0e9beba1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:23.656488 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1a136a-7673-417d-8644-20208c81ce71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:23.657117 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a136a-7673-417d-8644-20208c81ce71 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:23.657402 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a136a-7673-417d-8644-20208c81ce71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:23.657700 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a136a-7673-417d-8644-20208c81ce71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:23.659482 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1a136a-7673-417d-8644-20208c81ce71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:23.661057 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1a136a-7673-417d-8644-20208c81ce71 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:23.671734 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 809/1591] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:49:23.672672 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1a136a-7673-417d-8644-20208c81ce71 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:23.683931 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1a136a-7673-417d-8644-20208c81ce71 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:23.688714 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dac01c-9877-4b94-9ed3-95939e5f4b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:23.689387 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dac01c-9877-4b94-9ed3-95939e5f4b2a admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:23.689823 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dac01c-9877-4b94-9ed3-95939e5f4b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:23.690203 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dac01c-9877-4b94-9ed3-95939e5f4b2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:23.690908 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dac01c-9877-4b94-9ed3-95939e5f4b2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:23.692588 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dac01c-9877-4b94-9ed3-95939e5f4b2a admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:23.702661 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 810/1592] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:49:23.703570 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dac01c-9877-4b94-9ed3-95939e5f4b2a admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:23.706754 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-d0dac01c-9877-4b94-9ed3-95939e5f4b2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:23.719437 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 783/1593] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:23.724878 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994d5d6-f19e-4af0-8a07-bcd0e9beba1b None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:23.731529 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994d5d6-f19e-4af0-8a07-bcd0e9beba1b None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:23.737314 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f42fce-e0f4-4fde-851f-6b30ac5c4c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:23.738002 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f42fce-e0f4-4fde-851f-6b30ac5c4c61 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:23.738349 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f42fce-e0f4-4fde-851f-6b30ac5c4c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:23.738702 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f42fce-e0f4-4fde-851f-6b30ac5c4c61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:23.739455 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f42fce-e0f4-4fde-851f-6b30ac5c4c61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:23.741140 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f42fce-e0f4-4fde-851f-6b30ac5c4c61 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:23.751215 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 784/1594] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:49:23.751215 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f42fce-e0f4-4fde-851f-6b30ac5c4c61 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:23.757991 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:23.762906 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f42fce-e0f4-4fde-851f-6b30ac5c4c61 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:23.768160 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bfd9d5-9891-4977-a6dd-40aa236e2b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:23.768937 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bfd9d5-9891-4977-a6dd-40aa236e2b46 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:23.769215 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bfd9d5-9891-4977-a6dd-40aa236e2b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:23.769487 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bfd9d5-9891-4977-a6dd-40aa236e2b46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:23.770120 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bfd9d5-9891-4977-a6dd-40aa236e2b46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:23.771837 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bfd9d5-9891-4977-a6dd-40aa236e2b46 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:23.779700 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dac01c-9877-4b94-9ed3-95939e5f4b2a admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:23.783916 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8d82d-ec01-4f22-b1cc-ab198bd61e29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:23.788580 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8d82d-ec01-4f22-b1cc-ab198bd61e29 service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:23.788580 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8d82d-ec01-4f22-b1cc-ab198bd61e29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:23.788580 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8d82d-ec01-4f22-b1cc-ab198bd61e29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:23.788854 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 785/1595] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:49:23.788854 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bfd9d5-9891-4977-a6dd-40aa236e2b46 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:23.794412 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c8d82d-ec01-4f22-b1cc-ab198bd61e29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:23.796003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c8d82d-ec01-4f22-b1cc-ab198bd61e29 service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:23.802918 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bfd9d5-9891-4977-a6dd-40aa236e2b46 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:23.805417 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 811/1596] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:49:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:49:23.806194 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8d82d-ec01-4f22-b1cc-ab198bd61e29 service neutron] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:23.809071 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b05bbc-10f5-4126-b947-72cec4ca085b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:23.809676 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b05bbc-10f5-4126-b947-72cec4ca085b admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:23.809958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b05bbc-10f5-4126-b947-72cec4ca085b admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:23.810430 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b05bbc-10f5-4126-b947-72cec4ca085b admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:23.810931 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b05bbc-10f5-4126-b947-72cec4ca085b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:23.812881 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b05bbc-10f5-4126-b947-72cec4ca085b admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:23.816357 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8d82d-ec01-4f22-b1cc-ab198bd61e29 service neutron] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:23.821153 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cfc26b-3366-47a3-a089-00f28e7f58e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:23.822980 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cfc26b-3366-47a3-a089-00f28e7f58e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:23.822980 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cfc26b-3366-47a3-a089-00f28e7f58e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:23.822980 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cfc26b-3366-47a3-a089-00f28e7f58e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:23.822980 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cfc26b-3366-47a3-a089-00f28e7f58e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:23.824620 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cfc26b-3366-47a3-a089-00f28e7f58e0 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:23.830329 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 786/1597] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:49:23.831700 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b05bbc-10f5-4126-b947-72cec4ca085b admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:23.846354 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 812/1598] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:49:23.846354 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cfc26b-3366-47a3-a089-00f28e7f58e0 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:23.846650 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b05bbc-10f5-4126-b947-72cec4ca085b admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:23.851196 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91445afb-ffb8-4973-b7df-a25ef233accb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:23.852173 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cfc26b-3366-47a3-a089-00f28e7f58e0 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:23.852332 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91445afb-ffb8-4973-b7df-a25ef233accb admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:23.852610 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91445afb-ffb8-4973-b7df-a25ef233accb admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:23.853018 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91445afb-ffb8-4973-b7df-a25ef233accb admin admin] PATH_INFO: `/v3/projects/95c05358d0df40c5971e3822352eb8b9/users/3d49169ba293429da881db9296301f32/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:23.856830 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ac591-6436-4a1f-be96-202b8e83bf8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:23.857422 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ac591-6436-4a1f-be96-202b8e83bf8b admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:23.857711 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ac591-6436-4a1f-be96-202b8e83bf8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:23.857992 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ac591-6436-4a1f-be96-202b8e83bf8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:23.858625 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931ac591-6436-4a1f-be96-202b8e83bf8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:23.860258 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931ac591-6436-4a1f-be96-202b8e83bf8b admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:23.870271 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91445afb-ffb8-4973-b7df-a25ef233accb admin admin] RBAC: Authorizing `identity:create_grant(project_id=95c05358d0df40c5971e3822352eb8b9, user_id=3d49169ba293429da881db9296301f32, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:23.874720 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 813/1599] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:49:23.874720 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ac591-6436-4a1f-be96-202b8e83bf8b admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:23.875005 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91445afb-ffb8-4973-b7df-a25ef233accb admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:23.930577 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ac591-6436-4a1f-be96-202b8e83bf8b admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:23.934263 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f36db-c5dd-463c-b2d4-abb6c0f0f554 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:23.934869 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f36db-c5dd-463c-b2d4-abb6c0f0f554 service cinder] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:23.935148 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f36db-c5dd-463c-b2d4-abb6c0f0f554 service cinder] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:23.935457 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f36db-c5dd-463c-b2d4-abb6c0f0f554 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:23.944265 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6f36db-c5dd-463c-b2d4-abb6c0f0f554 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:23.945684 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6f36db-c5dd-463c-b2d4-abb6c0f0f554 service cinder] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:23.955410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 814/1600] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:49:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:49:23.956277 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f36db-c5dd-463c-b2d4-abb6c0f0f554 service cinder] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:23.960118 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 787/1601] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:49:23 2022] PUT /identity/v3/projects/95c05358d0df40c5971e3822352eb8b9/users/3d49169ba293429da881db9296301f32/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:49:23.961161 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91445afb-ffb8-4973-b7df-a25ef233accb admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:23.967274 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f36db-c5dd-463c-b2d4-abb6c0f0f554 service cinder] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:24.027486 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:24.032332 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c73737-c276-47cf-b6e1-18a102eca193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:24.033259 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c73737-c276-47cf-b6e1-18a102eca193 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:24.033639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c73737-c276-47cf-b6e1-18a102eca193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:24.033966 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c73737-c276-47cf-b6e1-18a102eca193 admin admin] PATH_INFO: `/v3/projects/cc40d256f0e24050b6b53fdbcbce1962/users/bd24d209dc9644e8b2860a0e676acd3b/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:24.041982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91445afb-ffb8-4973-b7df-a25ef233accb admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:24.047582 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0faf90-6e70-42f3-9503-18092fd75e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:24.048125 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0faf90-6e70-42f3-9503-18092fd75e43 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:24.048608 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0faf90-6e70-42f3-9503-18092fd75e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:24.052953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0faf90-6e70-42f3-9503-18092fd75e43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:24.052953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0faf90-6e70-42f3-9503-18092fd75e43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:24.053346 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c73737-c276-47cf-b6e1-18a102eca193 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc40d256f0e24050b6b53fdbcbce1962, user_id=bd24d209dc9644e8b2860a0e676acd3b, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:24.053472 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0faf90-6e70-42f3-9503-18092fd75e43 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:24.053571 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c73737-c276-47cf-b6e1-18a102eca193 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:24.062917 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 788/1602] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:49:24.063756 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0faf90-6e70-42f3-9503-18092fd75e43 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:24.076238 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0faf90-6e70-42f3-9503-18092fd75e43 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:24.081079 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b20950-aec7-4fd9-bcb8-010e76febd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:24.081748 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b20950-aec7-4fd9-bcb8-010e76febd91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:24.083301 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b20950-aec7-4fd9-bcb8-010e76febd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:24.083893 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b20950-aec7-4fd9-bcb8-010e76febd91 admin admin] PATH_INFO: `/v3/projects/0e20f793bd0b46a59df730183818d9f8/users/676b2a6bbf6e4df1b47dba2a9b8e8862/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:24.100057 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b20950-aec7-4fd9-bcb8-010e76febd91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e20f793bd0b46a59df730183818d9f8, user_id=676b2a6bbf6e4df1b47dba2a9b8e8862, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:24.102672 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b20950-aec7-4fd9-bcb8-010e76febd91 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:24.137859 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 815/1603] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:49:23 2022] PUT /identity/v3/projects/cc40d256f0e24050b6b53fdbcbce1962/users/bd24d209dc9644e8b2860a0e676acd3b/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:49:24.138974 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c73737-c276-47cf-b6e1-18a102eca193 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:24.198715 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:24.217785 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 789/1604] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:49:24 2022] PUT /identity/v3/projects/0e20f793bd0b46a59df730183818d9f8/users/676b2a6bbf6e4df1b47dba2a9b8e8862/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:49:24.217785 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b20950-aec7-4fd9-bcb8-010e76febd91 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:24.218018 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c73737-c276-47cf-b6e1-18a102eca193 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:24.284369 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680d16a-dbfa-4130-836f-7ce0ea7b2695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:24.285066 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680d16a-dbfa-4130-836f-7ce0ea7b2695 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:24.285346 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680d16a-dbfa-4130-836f-7ce0ea7b2695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:24.286248 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680d16a-dbfa-4130-836f-7ce0ea7b2695 admin admin] PATH_INFO: `/v3/projects/b1206a741c38411ab2498aec83388dfe/users/49bc3ec9f78445f996378f9ddf4f928b/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:24.296514 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b20950-aec7-4fd9-bcb8-010e76febd91 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:24.301769 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fcbff1-f33e-4e65-867e-62a8d1579f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:24.302425 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a680d16a-dbfa-4130-836f-7ce0ea7b2695 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1206a741c38411ab2498aec83388dfe, user_id=49bc3ec9f78445f996378f9ddf4f928b, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:24.302543 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcbff1-f33e-4e65-867e-62a8d1579f43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:24.302711 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcbff1-f33e-4e65-867e-62a8d1579f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:24.303092 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcbff1-f33e-4e65-867e-62a8d1579f43 admin admin] PATH_INFO: `/v3/projects/321f40f556444cef9a3bd080792042f5/users/82da1ca07d9445bd824e9de45ec6302a/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:24.305349 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a680d16a-dbfa-4130-836f-7ce0ea7b2695 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:24.320374 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fcbff1-f33e-4e65-867e-62a8d1579f43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=321f40f556444cef9a3bd080792042f5, user_id=82da1ca07d9445bd824e9de45ec6302a, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:24.323138 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fcbff1-f33e-4e65-867e-62a8d1579f43 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:24.381568 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 816/1605] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:49:24 2022] PUT /identity/v3/projects/b1206a741c38411ab2498aec83388dfe/users/49bc3ec9f78445f996378f9ddf4f928b/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:49:24.391691 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b1f779-6e2e-4e3a-8889-2a86f0a904a3 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:24.391691 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b1f779-6e2e-4e3a-8889-2a86f0a904a3 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:24.391691 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b1f779-6e2e-4e3a-8889-2a86f0a904a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:24.420250 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-72b1f779-6e2e-4e3a-8889-2a86f0a904a3 None None] MFA Rules not processed for user `3d49169ba293429da881db9296301f32`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:24.420250 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-72b1f779-6e2e-4e3a-8889-2a86f0a904a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:24.444611 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 790/1606] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:49:24 2022] PUT /identity/v3/projects/321f40f556444cef9a3bd080792042f5/users/82da1ca07d9445bd824e9de45ec6302a/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:49:24.453393 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fcbff1-f33e-4e65-867e-62a8d1579f43 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:24.495259 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 817/1607] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:24.503003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540afdc-05c4-4cfb-8470-c2c8c77f9c90 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:24.503003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540afdc-05c4-4cfb-8470-c2c8c77f9c90 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:24.503003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540afdc-05c4-4cfb-8470-c2c8c77f9c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:24.528292 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-5540afdc-05c4-4cfb-8470-c2c8c77f9c90 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:24.528416 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fcbff1-f33e-4e65-867e-62a8d1579f43 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:24.529645 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-5540afdc-05c4-4cfb-8470-c2c8c77f9c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:24.534091 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d113fd3-1a5c-46ce-9e9e-d528d2feb88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:24.534860 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d113fd3-1a5c-46ce-9e9e-d528d2feb88a admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:24.535196 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d113fd3-1a5c-46ce-9e9e-d528d2feb88a admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:24.535563 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d113fd3-1a5c-46ce-9e9e-d528d2feb88a admin admin] PATH_INFO: `/v3/projects/d7994e24004a42ac88491cf70b3d1086/users/f938505064304ae19cd0a6df370b7721/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:24.540495 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 818/1608] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:24.541931 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06931409-4422-453c-b451-e55baeee4c5d None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:24.542221 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06931409-4422-453c-b451-e55baeee4c5d None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:24.542648 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06931409-4422-453c-b451-e55baeee4c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:24.556758 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d113fd3-1a5c-46ce-9e9e-d528d2feb88a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7994e24004a42ac88491cf70b3d1086, user_id=f938505064304ae19cd0a6df370b7721, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:24.559592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d113fd3-1a5c-46ce-9e9e-d528d2feb88a admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:24.567666 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-06931409-4422-453c-b451-e55baeee4c5d None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:24.569045 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-06931409-4422-453c-b451-e55baeee4c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:24.576950 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 819/1609] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:24.580357 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9739f1-2fa8-4c29-94b9-14ec3986b26e None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:24.580357 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9739f1-2fa8-4c29-94b9-14ec3986b26e None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:24.580357 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9739f1-2fa8-4c29-94b9-14ec3986b26e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:24.612350 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-ef9739f1-2fa8-4c29-94b9-14ec3986b26e None None] MFA Rules not processed for user `49bc3ec9f78445f996378f9ddf4f928b`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:24.613578 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-ef9739f1-2fa8-4c29-94b9-14ec3986b26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:24.657279 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:24.683368 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 791/1610] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:49:24 2022] PUT /identity/v3/projects/d7994e24004a42ac88491cf70b3d1086/users/f938505064304ae19cd0a6df370b7721/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:49:24.687512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 820/1611] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:24.687648 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864e38a-3984-4934-8919-15ac62426002 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:24.687648 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864e38a-3984-4934-8919-15ac62426002 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:24.687648 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864e38a-3984-4934-8919-15ac62426002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:24.688983 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27842489-2d43-43ef-9497-bcfe3ac8edc5 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:24.690445 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27842489-2d43-43ef-9497-bcfe3ac8edc5 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:24.690734 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27842489-2d43-43ef-9497-bcfe3ac8edc5 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:24.692326 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 821/1612] 10.222.0.33 () {54 vars in 961 bytes} [Mon Oct 3 12:49:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:49:24.697054 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27842489-2d43-43ef-9497-bcfe3ac8edc5 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:24.724227 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-2864e38a-3984-4934-8919-15ac62426002 None None] MFA Rules not processed for user `82da1ca07d9445bd824e9de45ec6302a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:24.725467 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-2864e38a-3984-4934-8919-15ac62426002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:24.739981 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:24.769760 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27842489-2d43-43ef-9497-bcfe3ac8edc5 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:24.774801 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6541fa-53c3-48bf-bd92-c325bc4e58e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:24.775509 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6541fa-53c3-48bf-bd92-c325bc4e58e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:24.775742 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6541fa-53c3-48bf-bd92-c325bc4e58e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:24.776014 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6541fa-53c3-48bf-bd92-c325bc4e58e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:24.799412 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 792/1613] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:24.800261 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864e38a-3984-4934-8919-15ac62426002 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:24.816261 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864e38a-3984-4934-8919-15ac62426002 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:24.820673 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da645be-e958-4a8a-ae71-bd6d3f896c9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:24.821368 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da645be-e958-4a8a-ae71-bd6d3f896c9a service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:24.821604 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da645be-e958-4a8a-ae71-bd6d3f896c9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:24.821878 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da645be-e958-4a8a-ae71-bd6d3f896c9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:24.850854 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6541fa-53c3-48bf-bd92-c325bc4e58e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:24.852330 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6541fa-53c3-48bf-bd92-c325bc4e58e5 service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:24.863081 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 822/1614] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:49:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:49:24.869617 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c320d-62d7-4751-abd1-8cb0a230cc58 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:24.869617 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c320d-62d7-4751-abd1-8cb0a230cc58 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:24.869617 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c320d-62d7-4751-abd1-8cb0a230cc58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:24.885022 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-227c320d-62d7-4751-abd1-8cb0a230cc58 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:24.886612 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-227c320d-62d7-4751-abd1-8cb0a230cc58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:24.890385 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da645be-e958-4a8a-ae71-bd6d3f896c9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:24.892220 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da645be-e958-4a8a-ae71-bd6d3f896c9a service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:24.898726 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 823/1615] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:24.898726 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c320d-62d7-4751-abd1-8cb0a230cc58 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:24.903041 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 793/1616] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:49:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:49:24.904246 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467cf87-113f-43aa-a9af-83e8ed0a3d72 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:24.904525 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467cf87-113f-43aa-a9af-83e8ed0a3d72 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:24.904796 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467cf87-113f-43aa-a9af-83e8ed0a3d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:24.906328 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c320d-62d7-4751-abd1-8cb0a230cc58 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:24.912529 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461014ac-a521-4a79-baab-b8d51c332976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:24.913155 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461014ac-a521-4a79-baab-b8d51c332976 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:24.913484 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461014ac-a521-4a79-baab-b8d51c332976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:24.913806 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461014ac-a521-4a79-baab-b8d51c332976 admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:24.914500 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461014ac-a521-4a79-baab-b8d51c332976 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:24.925774 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461014ac-a521-4a79-baab-b8d51c332976 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:24.957062 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-2467cf87-113f-43aa-a9af-83e8ed0a3d72 None None] MFA Rules not processed for user `990f5837994f4a088c656c9b33e9a0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:24.958621 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-2467cf87-113f-43aa-a9af-83e8ed0a3d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:25.050875 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 794/1617] 10.222.0.33 () {58 vars in 1058 bytes} [Mon Oct 3 12:49:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:25.052300 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde156c4-034d-4416-a21b-778b1b5de179 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:25.052758 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde156c4-034d-4416-a21b-778b1b5de179 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:25.053078 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde156c4-034d-4416-a21b-778b1b5de179 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:25.054664 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 795/1618] 10.222.0.33 () {54 vars in 961 bytes} [Mon Oct 3 12:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:49:25.060010 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde156c4-034d-4416-a21b-778b1b5de179 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:25.078490 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 824/1619] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:49:24 2022] POST /identity/v3/projects => generated 379 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:49:25.078490 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1bd4b9-b05b-4eac-91e0-73e8b96b914f None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:25.078490 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1bd4b9-b05b-4eac-91e0-73e8b96b914f None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:25.078490 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1bd4b9-b05b-4eac-91e0-73e8b96b914f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:25.105550 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde156c4-034d-4416-a21b-778b1b5de179 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:25.108370 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-bc1bd4b9-b05b-4eac-91e0-73e8b96b914f None None] MFA Rules not processed for user `990f5837994f4a088c656c9b33e9a0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:25.109977 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-bc1bd4b9-b05b-4eac-91e0-73e8b96b914f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:25.172291 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bad85cd-ac1b-4722-8d84-6858658a7351 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:25.172982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad85cd-ac1b-4722-8d84-6858658a7351 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:25.173178 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad85cd-ac1b-4722-8d84-6858658a7351 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:25.173591 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad85cd-ac1b-4722-8d84-6858658a7351 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:25.257455 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 825/1620] 10.222.0.33 () {58 vars in 1058 bytes} [Mon Oct 3 12:49:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:25.258383 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1bd4b9-b05b-4eac-91e0-73e8b96b914f None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:25.273933 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bad85cd-ac1b-4722-8d84-6858658a7351 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:25.275916 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bad85cd-ac1b-4722-8d84-6858658a7351 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:25.287048 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1bd4b9-b05b-4eac-91e0-73e8b96b914f None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:25.293951 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 796/1621] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:49:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:49:25.296121 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bad85cd-ac1b-4722-8d84-6858658a7351 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:25.297965 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369daa3-ee26-4877-be9c-106a53e68498 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:25.298827 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369daa3-ee26-4877-be9c-106a53e68498 service nova] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:25.299656 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369daa3-ee26-4877-be9c-106a53e68498 service nova] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:25.300214 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369daa3-ee26-4877-be9c-106a53e68498 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:25.308496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bad85cd-ac1b-4722-8d84-6858658a7351 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:25.315347 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6a18e-9114-45f6-a7f4-ca4c8b0589e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:25.315581 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6a18e-9114-45f6-a7f4-ca4c8b0589e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:25.315975 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6a18e-9114-45f6-a7f4-ca4c8b0589e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:25.316258 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6a18e-9114-45f6-a7f4-ca4c8b0589e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:25.317004 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae6a18e-9114-45f6-a7f4-ca4c8b0589e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:25.319422 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae6a18e-9114-45f6-a7f4-ca4c8b0589e6 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:25.378550 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369daa3-ee26-4877-be9c-106a53e68498 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:25.380809 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369daa3-ee26-4877-be9c-106a53e68498 service nova] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:25.397855 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 826/1622] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:49:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:49:25.399115 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369daa3-ee26-4877-be9c-106a53e68498 service nova] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:25.407614 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:25.418518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369daa3-ee26-4877-be9c-106a53e68498 service nova] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:25.444824 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 797/1623] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:49:25 2022] POST /identity/v3/users => generated 296 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:49:25.449227 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6a18e-9114-45f6-a7f4-ca4c8b0589e6 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:25.453286 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ab712-1ba2-45d5-82a9-2fa3443bf692 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:25.455913 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ab712-1ba2-45d5-82a9-2fa3443bf692 service nova] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:25.456237 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ab712-1ba2-45d5-82a9-2fa3443bf692 service nova] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:25.458535 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ab712-1ba2-45d5-82a9-2fa3443bf692 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:25.497128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6a18e-9114-45f6-a7f4-ca4c8b0589e6 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:25.525786 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070cfbdf-d86b-4d33-a264-2f7f5e8f15d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:25.526785 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070cfbdf-d86b-4d33-a264-2f7f5e8f15d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:25.528580 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070cfbdf-d86b-4d33-a264-2f7f5e8f15d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:25.529173 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070cfbdf-d86b-4d33-a264-2f7f5e8f15d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:25.530981 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070cfbdf-d86b-4d33-a264-2f7f5e8f15d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:25.540491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070cfbdf-d86b-4d33-a264-2f7f5e8f15d3 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:25.569023 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:25.580971 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 798/1624] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:49:25.584831 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070cfbdf-d86b-4d33-a264-2f7f5e8f15d3 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:25.609465 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070cfbdf-d86b-4d33-a264-2f7f5e8f15d3 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:25.623493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaae491-7faa-4679-97de-0adad6f448e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:25.624186 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaae491-7faa-4679-97de-0adad6f448e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:25.624473 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaae491-7faa-4679-97de-0adad6f448e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:25.624749 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaae491-7faa-4679-97de-0adad6f448e5 admin admin] PATH_INFO: `/v3/projects/9661cd7492664918b8bd60a504c97ccd/users/ca98ae0f1ae2429bb37c54e9c2b12de5/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:25.652940 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ab712-1ba2-45d5-82a9-2fa3443bf692 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:25.655285 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ab712-1ba2-45d5-82a9-2fa3443bf692 service nova] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:25.666813 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaae491-7faa-4679-97de-0adad6f448e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9661cd7492664918b8bd60a504c97ccd, user_id=ca98ae0f1ae2429bb37c54e9c2b12de5, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:25.670674 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaae491-7faa-4679-97de-0adad6f448e5 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:25.682825 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 827/1625] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:49:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:49:25.751146 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 799/1626] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:49:25 2022] PUT /identity/v3/projects/9661cd7492664918b8bd60a504c97ccd/users/ca98ae0f1ae2429bb37c54e9c2b12de5/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:49:25.756230 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ab712-1ba2-45d5-82a9-2fa3443bf692 service nova] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:25.773050 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ab712-1ba2-45d5-82a9-2fa3443bf692 service nova] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:25.926026 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf453ab-9dc9-40d9-a003-1bef5882f945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:25.926026 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf453ab-9dc9-40d9-a003-1bef5882f945 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:25.926026 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf453ab-9dc9-40d9-a003-1bef5882f945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:25.926026 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf453ab-9dc9-40d9-a003-1bef5882f945 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:25.929573 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac240826-721c-4992-a578-822edcc64ad6 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:25.929871 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf453ab-9dc9-40d9-a003-1bef5882f945 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:25.930207 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac240826-721c-4992-a578-822edcc64ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:25.930647 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac240826-721c-4992-a578-822edcc64ad6 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:25.932242 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf453ab-9dc9-40d9-a003-1bef5882f945 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:25.932600 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 800/1627] 10.222.0.33 () {54 vars in 912 bytes} [Mon Oct 3 12:49:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:49:25.938446 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6aa3d-2336-4913-ae55-f96babbd7dd2 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:25.938917 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6aa3d-2336-4913-ae55-f96babbd7dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:25.939625 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6aa3d-2336-4913-ae55-f96babbd7dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:25.945626 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 828/1628] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:49:25.949485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf453ab-9dc9-40d9-a003-1bef5882f945 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:25.961343 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-81a6aa3d-2336-4913-ae55-f96babbd7dd2 None None] MFA Rules not processed for user `990f5837994f4a088c656c9b33e9a0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:25.961540 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf453ab-9dc9-40d9-a003-1bef5882f945 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:25.962868 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-81a6aa3d-2336-4913-ae55-f96babbd7dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:25.966306 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec2d54-eb81-44ba-a04a-998395f49f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:25.967078 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec2d54-eb81-44ba-a04a-998395f49f77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:25.967395 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec2d54-eb81-44ba-a04a-998395f49f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:25.967722 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec2d54-eb81-44ba-a04a-998395f49f77 admin admin] PATH_INFO: `/v3/projects/9661cd7492664918b8bd60a504c97ccd/users/ca98ae0f1ae2429bb37c54e9c2b12de5/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:25.970142 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ec2d54-eb81-44ba-a04a-998395f49f77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9661cd7492664918b8bd60a504c97ccd, user_id=ca98ae0f1ae2429bb37c54e9c2b12de5, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:25.974722 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ec2d54-eb81-44ba-a04a-998395f49f77 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:26.006936 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 829/1629] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:49:25 2022] PUT /identity/v3/projects/9661cd7492664918b8bd60a504c97ccd/users/ca98ae0f1ae2429bb37c54e9c2b12de5/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:49:26.018737 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a6ffc-1f93-4d23-b273-7723662bbc2b None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:26.022016 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a6ffc-1f93-4d23-b273-7723662bbc2b None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:26.022016 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a6ffc-1f93-4d23-b273-7723662bbc2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:26.092352 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-3e2a6ffc-1f93-4d23-b273-7723662bbc2b None None] MFA Rules not processed for user `ca98ae0f1ae2429bb37c54e9c2b12de5`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:26.093590 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-3e2a6ffc-1f93-4d23-b273-7723662bbc2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:26.139112 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 801/1630] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Oct 3 12:49:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:26.140323 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e128c60-dae8-4905-8417-fe2d956c516a None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:26.140611 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e128c60-dae8-4905-8417-fe2d956c516a None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:26.140888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e128c60-dae8-4905-8417-fe2d956c516a None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:26.142375 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 802/1631] 10.222.0.33 () {54 vars in 912 bytes} [Mon Oct 3 12:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:49:26.144851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:26.158152 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e128c60-dae8-4905-8417-fe2d956c516a None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:26.224209 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:26.289187 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 830/1632] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:26 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:26.290735 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f5d1d-2f0a-41e6-b6e1-d1147e0d21c0 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:26.291402 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f5d1d-2f0a-41e6-b6e1-d1147e0d21c0 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:26.291402 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f5d1d-2f0a-41e6-b6e1-d1147e0d21c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:26.313930 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-548f5d1d-2f0a-41e6-b6e1-d1147e0d21c0 None None] MFA Rules not processed for user `990f5837994f4a088c656c9b33e9a0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:26.315909 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-548f5d1d-2f0a-41e6-b6e1-d1147e0d21c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:26.325951 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e128c60-dae8-4905-8417-fe2d956c516a None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:26.332584 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8f94b7-a689-47ec-bc51-868363c242d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:26.333214 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8f94b7-a689-47ec-bc51-868363c242d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:26.333493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8f94b7-a689-47ec-bc51-868363c242d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:26.333797 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8f94b7-a689-47ec-bc51-868363c242d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:26.624822 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 831/1633] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Oct 3 12:49:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:26.624822 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f5d1d-2f0a-41e6-b6e1-d1147e0d21c0 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:26.624822 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f5d1d-2f0a-41e6-b6e1-d1147e0d21c0 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:26.624822 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd14d9-18bb-4eab-865f-0247565b5476 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:26.626525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd14d9-18bb-4eab-865f-0247565b5476 service cinder] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:26.626525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd14d9-18bb-4eab-865f-0247565b5476 service cinder] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:26.626525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd14d9-18bb-4eab-865f-0247565b5476 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:26.644151 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8f94b7-a689-47ec-bc51-868363c242d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:26.644151 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8f94b7-a689-47ec-bc51-868363c242d0 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:26.645459 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bd14d9-18bb-4eab-865f-0247565b5476 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:26.645459 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bd14d9-18bb-4eab-865f-0247565b5476 service cinder] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:26.645750 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 803/1634] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:49:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:49:26.646243 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 832/1635] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:49:26 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:49:27.211182 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8f94b7-a689-47ec-bc51-868363c242d0 service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:27.221549 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8f94b7-a689-47ec-bc51-868363c242d0 service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:27.246041 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d9d76-3318-41a8-a125-28b12bc047c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:27.246791 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d9d76-3318-41a8-a125-28b12bc047c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:27.247178 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d9d76-3318-41a8-a125-28b12bc047c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:27.247577 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d9d76-3318-41a8-a125-28b12bc047c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:27.263379 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1d9d76-3318-41a8-a125-28b12bc047c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:27.266120 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1d9d76-3318-41a8-a125-28b12bc047c8 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:27.283479 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 804/1636] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:49:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:49:27.671706 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd14d9-18bb-4eab-865f-0247565b5476 service cinder] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:27.687373 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd14d9-18bb-4eab-865f-0247565b5476 service cinder] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:27.692740 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c74af-435c-4dd6-9f9f-8d7287ce2e8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:27.693366 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c74af-435c-4dd6-9f9f-8d7287ce2e8c service cinder] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:27.693819 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c74af-435c-4dd6-9f9f-8d7287ce2e8c service cinder] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:27.694103 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c74af-435c-4dd6-9f9f-8d7287ce2e8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:27.788324 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c74af-435c-4dd6-9f9f-8d7287ce2e8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:27.789757 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c74af-435c-4dd6-9f9f-8d7287ce2e8c service cinder] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:27.804403 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 833/1637] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:49:27 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:49:28.829175 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d9d76-3318-41a8-a125-28b12bc047c8 service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:28.965213 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d9d76-3318-41a8-a125-28b12bc047c8 service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:28.971243 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b34a675-fcaa-479d-8aac-3608c9956609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:28.972548 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34a675-fcaa-479d-8aac-3608c9956609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:28.973109 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34a675-fcaa-479d-8aac-3608c9956609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:28.973642 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34a675-fcaa-479d-8aac-3608c9956609 admin admin] PATH_INFO: `/v3/users/82da1ca07d9445bd824e9de45ec6302a` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:28.975043 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b34a675-fcaa-479d-8aac-3608c9956609 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82da1ca07d9445bd824e9de45ec6302a)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:28.977470 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b34a675-fcaa-479d-8aac-3608c9956609 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:29.197263 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-5b34a675-fcaa-479d-8aac-3608c9956609 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82da1ca07d9445bd824e9de45ec6302a'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:29.198735 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:29.239386 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-5b34a675-fcaa-479d-8aac-3608c9956609 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82da1ca07d9445bd824e9de45ec6302a'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:29.240680 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-5b34a675-fcaa-479d-8aac-3608c9956609 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82da1ca07d9445bd824e9de45ec6302a'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:29.266613 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-5b34a675-fcaa-479d-8aac-3608c9956609 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82da1ca07d9445bd824e9de45ec6302a'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:29.362627 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-5b34a675-fcaa-479d-8aac-3608c9956609 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82da1ca07d9445bd824e9de45ec6302a'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:29.374007 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 805/1638] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:49:28 2022] DELETE /identity/v3/users/82da1ca07d9445bd824e9de45ec6302a => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:49:29.378590 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eae313-05a6-4bcb-a6f7-2dd641c54ade None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:29.379891 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eae313-05a6-4bcb-a6f7-2dd641c54ade None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:29.380375 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eae313-05a6-4bcb-a6f7-2dd641c54ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:29.410860 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-80eae313-05a6-4bcb-a6f7-2dd641c54ade None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:29.413624 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-80eae313-05a6-4bcb-a6f7-2dd641c54ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:29.436328 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:29.541709 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 834/1639] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:29.565057 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b34a675-fcaa-479d-8aac-3608c9956609 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:29.567480 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-5b34a675-fcaa-479d-8aac-3608c9956609 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:29.650814 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b34a675-fcaa-479d-8aac-3608c9956609 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:29.656008 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fccb58-f83e-437e-9c77-b74be2061214 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:29.656873 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fccb58-f83e-437e-9c77-b74be2061214 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:29.657321 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fccb58-f83e-437e-9c77-b74be2061214 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:29.658008 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fccb58-f83e-437e-9c77-b74be2061214 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:29.670629 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fccb58-f83e-437e-9c77-b74be2061214 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:29.672863 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fccb58-f83e-437e-9c77-b74be2061214 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:29.714764 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 806/1640] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:49:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:49:31.640795 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eae313-05a6-4bcb-a6f7-2dd641c54ade None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:31.642502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-80eae313-05a6-4bcb-a6f7-2dd641c54ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:31.661607 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eae313-05a6-4bcb-a6f7-2dd641c54ade None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:31.672824 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe8c215-bc92-4ca6-8296-6c5c3169cc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:31.675737 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8c215-bc92-4ca6-8296-6c5c3169cc72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:31.676211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8c215-bc92-4ca6-8296-6c5c3169cc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:31.678755 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8c215-bc92-4ca6-8296-6c5c3169cc72 admin admin] PATH_INFO: `/v3/projects/321f40f556444cef9a3bd080792042f5` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:31.690821 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe8c215-bc92-4ca6-8296-6c5c3169cc72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=321f40f556444cef9a3bd080792042f5)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:31.695619 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe8c215-bc92-4ca6-8296-6c5c3169cc72 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:31.912492 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-fbe8c215-bc92-4ca6-8296-6c5c3169cc72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '321f40f556444cef9a3bd080792042f5', 'request_id': 'req-fbe8c215-bc92-4ca6-8296-6c5c3169cc72'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:31.929003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-fbe8c215-bc92-4ca6-8296-6c5c3169cc72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '321f40f556444cef9a3bd080792042f5', 'request_id': 'req-fbe8c215-bc92-4ca6-8296-6c5c3169cc72'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:31.989298 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fccb58-f83e-437e-9c77-b74be2061214 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:31.991901 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-10fccb58-f83e-437e-9c77-b74be2061214 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:32.021238 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fccb58-f83e-437e-9c77-b74be2061214 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:32.054464 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-fbe8c215-bc92-4ca6-8296-6c5c3169cc72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '321f40f556444cef9a3bd080792042f5', 'request_id': 'req-fbe8c215-bc92-4ca6-8296-6c5c3169cc72'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:32.079899 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-fbe8c215-bc92-4ca6-8296-6c5c3169cc72 admin admin] The token cache is being invalidate because project 321f40f556444cef9a3bd080792042f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:49:32.080623 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-fbe8c215-bc92-4ca6-8296-6c5c3169cc72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:32.092428 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-fbe8c215-bc92-4ca6-8296-6c5c3169cc72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:32.099145 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 835/1641] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:49:31 2022] DELETE /identity/v3/projects/321f40f556444cef9a3bd080792042f5 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:49:32.105386 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf92b3-2b8c-4b63-bddf-3e43d0febed7 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:32.108431 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf92b3-2b8c-4b63-bddf-3e43d0febed7 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:32.108947 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf92b3-2b8c-4b63-bddf-3e43d0febed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:32.194353 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-f9cf92b3-2b8c-4b63-bddf-3e43d0febed7 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:32.195815 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-f9cf92b3-2b8c-4b63-bddf-3e43d0febed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:32.218879 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd3d421-f92c-4b9f-86a4-2e48a802d490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:32.219932 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3d421-f92c-4b9f-86a4-2e48a802d490 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:32.220949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3d421-f92c-4b9f-86a4-2e48a802d490 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:32.221375 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3d421-f92c-4b9f-86a4-2e48a802d490 admin admin] PATH_INFO: `/v3/users/3d49169ba293429da881db9296301f32` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:32.229274 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd3d421-f92c-4b9f-86a4-2e48a802d490 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d49169ba293429da881db9296301f32)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:32.231569 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd3d421-f92c-4b9f-86a4-2e48a802d490 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:32.377203 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 836/1642] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:32.381997 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e57b41-8098-4080-8f4a-8dd13f7bcc41 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:32.382411 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e57b41-8098-4080-8f4a-8dd13f7bcc41 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:32.382685 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e57b41-8098-4080-8f4a-8dd13f7bcc41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:32.452538 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-78e57b41-8098-4080-8f4a-8dd13f7bcc41 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:32.454507 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-78e57b41-8098-4080-8f4a-8dd13f7bcc41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:32.473037 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 837/1643] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:32.476286 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e57b41-8098-4080-8f4a-8dd13f7bcc41 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:32.526116 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e57b41-8098-4080-8f4a-8dd13f7bcc41 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:32.536232 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13842df8-f688-4254-a39d-07bfd267ef99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:32.541162 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13842df8-f688-4254-a39d-07bfd267ef99 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:32.541451 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13842df8-f688-4254-a39d-07bfd267ef99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:32.541732 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13842df8-f688-4254-a39d-07bfd267ef99 admin admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:32.542415 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13842df8-f688-4254-a39d-07bfd267ef99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:49:32.542725 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13842df8-f688-4254-a39d-07bfd267ef99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:32.543099 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1bd3d421-f92c-4b9f-86a4-2e48a802d490 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d49169ba293429da881db9296301f32'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:32.546019 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13842df8-f688-4254-a39d-07bfd267ef99 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:32.574385 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1bd3d421-f92c-4b9f-86a4-2e48a802d490 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d49169ba293429da881db9296301f32'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:32.575385 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1bd3d421-f92c-4b9f-86a4-2e48a802d490 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d49169ba293429da881db9296301f32'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:32.581894 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 838/1644] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:49:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:49:32.583092 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13842df8-f688-4254-a39d-07bfd267ef99 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:32.589949 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-13842df8-f688-4254-a39d-07bfd267ef99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:32.598487 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:32.607375 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1bd3d421-f92c-4b9f-86a4-2e48a802d490 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d49169ba293429da881db9296301f32'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:32.617937 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13842df8-f688-4254-a39d-07bfd267ef99 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:32.643291 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ff7fd-f1e3-41e8-939f-fb0cf69cee94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:32.644149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ff7fd-f1e3-41e8-939f-fb0cf69cee94 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:32.644443 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ff7fd-f1e3-41e8-939f-fb0cf69cee94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:32.644708 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ff7fd-f1e3-41e8-939f-fb0cf69cee94 admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:32.645353 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563ff7fd-f1e3-41e8-939f-fb0cf69cee94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:32.647270 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563ff7fd-f1e3-41e8-939f-fb0cf69cee94 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:32.701077 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1bd3d421-f92c-4b9f-86a4-2e48a802d490 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d49169ba293429da881db9296301f32'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:32.722050 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 807/1645] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:49:31 2022] DELETE /identity/v3/users/3d49169ba293429da881db9296301f32 => generated 0 bytes in 729 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:49:32.723611 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd733619-1770-4652-a50f-5a49ba16aba1 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:32.723896 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd733619-1770-4652-a50f-5a49ba16aba1 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:32.725223 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd733619-1770-4652-a50f-5a49ba16aba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:32.781363 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:32.781572 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-dd733619-1770-4652-a50f-5a49ba16aba1 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:32.784508 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-dd733619-1770-4652-a50f-5a49ba16aba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:32.798843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 839/1646] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:49:32 2022] POST /identity/v3/projects => generated 367 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:49:32.804734 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ff7fd-f1e3-41e8-939f-fb0cf69cee94 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:32.980961 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 808/1647] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:33.030414 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ff7fd-f1e3-41e8-939f-fb0cf69cee94 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:33.050526 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33c2b6-4628-4210-8e7c-40cdb8eb23d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:33.051299 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33c2b6-4628-4210-8e7c-40cdb8eb23d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:33.051683 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33c2b6-4628-4210-8e7c-40cdb8eb23d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:33.052167 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33c2b6-4628-4210-8e7c-40cdb8eb23d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:33.056198 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd33c2b6-4628-4210-8e7c-40cdb8eb23d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:33.064501 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd33c2b6-4628-4210-8e7c-40cdb8eb23d0 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:33.076845 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd733619-1770-4652-a50f-5a49ba16aba1 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:33.081694 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-dd733619-1770-4652-a50f-5a49ba16aba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:33.261387 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 840/1648] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:49:32 2022] POST /identity/v3/users => generated 284 bytes in 459 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:49:33.267002 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33c2b6-4628-4210-8e7c-40cdb8eb23d0 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:33.319708 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33c2b6-4628-4210-8e7c-40cdb8eb23d0 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:33.319895 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd733619-1770-4652-a50f-5a49ba16aba1 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:33.328994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e77cc1-0909-4cc1-86e8-f641754e06f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:33.329795 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e77cc1-0909-4cc1-86e8-f641754e06f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:33.346899 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e77cc1-0909-4cc1-86e8-f641754e06f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:33.347178 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e77cc1-0909-4cc1-86e8-f641754e06f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:33.349461 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e77cc1-0909-4cc1-86e8-f641754e06f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:33.349809 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e77cc1-0909-4cc1-86e8-f641754e06f1 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:33.381822 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 841/1649] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:49:33.382834 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e77cc1-0909-4cc1-86e8-f641754e06f1 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:33.408934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e77cc1-0909-4cc1-86e8-f641754e06f1 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:33.428372 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c48aea-1fa2-4b3b-90a8-58645beb158f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:33.429294 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c48aea-1fa2-4b3b-90a8-58645beb158f admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:33.429579 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c48aea-1fa2-4b3b-90a8-58645beb158f admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:33.431450 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c48aea-1fa2-4b3b-90a8-58645beb158f admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:33.432118 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c48aea-1fa2-4b3b-90a8-58645beb158f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:33.442466 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c48aea-1fa2-4b3b-90a8-58645beb158f admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:33.493412 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 842/1650] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:49:33.494301 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c48aea-1fa2-4b3b-90a8-58645beb158f admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:33.506826 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c48aea-1fa2-4b3b-90a8-58645beb158f admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:33.512200 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c089391-dc4a-4b50-8747-9edc176158f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:33.514944 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c089391-dc4a-4b50-8747-9edc176158f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:33.515485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c089391-dc4a-4b50-8747-9edc176158f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:33.516063 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c089391-dc4a-4b50-8747-9edc176158f1 admin admin] PATH_INFO: `/v3/projects/02b778cf9c9b40eba64364a8505a0b94/users/da86b18305834c19a0b7002feaad2921/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:33.547726 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf98b36-fc3e-47e4-a43d-afd02abbab80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:33.549180 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf98b36-fc3e-47e4-a43d-afd02abbab80 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:33.553179 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf98b36-fc3e-47e4-a43d-afd02abbab80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:33.555530 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf98b36-fc3e-47e4-a43d-afd02abbab80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:33.560128 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c089391-dc4a-4b50-8747-9edc176158f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02b778cf9c9b40eba64364a8505a0b94, user_id=da86b18305834c19a0b7002feaad2921, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:33.565943 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c089391-dc4a-4b50-8747-9edc176158f1 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:33.581282 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf98b36-fc3e-47e4-a43d-afd02abbab80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:33.582967 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf98b36-fc3e-47e4-a43d-afd02abbab80 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:33.616176 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 809/1651] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:49:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:49:33.655862 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 843/1652] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:49:33 2022] PUT /identity/v3/projects/02b778cf9c9b40eba64364a8505a0b94/users/da86b18305834c19a0b7002feaad2921/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:49:33.658877 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4424c-e621-4732-8942-72ce7bd2fb39 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:33.659661 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4424c-e621-4732-8942-72ce7bd2fb39 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:33.660092 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4424c-e621-4732-8942-72ce7bd2fb39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:33.718076 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-83c4424c-e621-4732-8942-72ce7bd2fb39 None None] MFA Rules not processed for user `da86b18305834c19a0b7002feaad2921`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:33.723445 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-83c4424c-e621-4732-8942-72ce7bd2fb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:33.905828 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 810/1653] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:49:33.914125 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c089391-dc4a-4b50-8747-9edc176158f1 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:33.929690 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:34.104240 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c089391-dc4a-4b50-8747-9edc176158f1 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:34.123981 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c452fb72-dca6-4100-a861-b1688ed136fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:34.124675 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452fb72-dca6-4100-a861-b1688ed136fe admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:34.124986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452fb72-dca6-4100-a861-b1688ed136fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:34.125368 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452fb72-dca6-4100-a861-b1688ed136fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:34.126179 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c452fb72-dca6-4100-a861-b1688ed136fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:34.131242 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c452fb72-dca6-4100-a861-b1688ed136fe admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:34.259598 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 844/1654] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:49:33 2022] POST /identity/v3/projects => generated 369 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:49:34.268975 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4424c-e621-4732-8942-72ce7bd2fb39 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:34.290598 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4424c-e621-4732-8942-72ce7bd2fb39 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:34.432508 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c13945-30db-486e-8226-19c3cd002729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:34.433147 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c13945-30db-486e-8226-19c3cd002729 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:34.433423 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c13945-30db-486e-8226-19c3cd002729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:34.433820 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c13945-30db-486e-8226-19c3cd002729 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:34.434307 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c13945-30db-486e-8226-19c3cd002729 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:34.436255 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c13945-30db-486e-8226-19c3cd002729 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:34.547959 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 811/1655] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:49:34 2022] POST /identity/v3/users => generated 291 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:49:34.552517 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c452fb72-dca6-4100-a861-b1688ed136fe admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:34.575787 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c452fb72-dca6-4100-a861-b1688ed136fe admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:34.588745 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de9689-48d9-4ef4-a3e1-8487e7de0c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:34.591046 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de9689-48d9-4ef4-a3e1-8487e7de0c8f admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:34.592777 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de9689-48d9-4ef4-a3e1-8487e7de0c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:34.593205 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de9689-48d9-4ef4-a3e1-8487e7de0c8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:34.594162 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de9689-48d9-4ef4-a3e1-8487e7de0c8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:34.600880 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de9689-48d9-4ef4-a3e1-8487e7de0c8f admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:34.633369 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 845/1656] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:49:34.642946 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c13945-30db-486e-8226-19c3cd002729 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:34.689735 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c13945-30db-486e-8226-19c3cd002729 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:34.702227 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dee1594-2496-4ae9-8e4d-8daa977e5999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:34.706379 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee1594-2496-4ae9-8e4d-8daa977e5999 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:34.707912 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee1594-2496-4ae9-8e4d-8daa977e5999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:34.710195 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee1594-2496-4ae9-8e4d-8daa977e5999 admin admin] PATH_INFO: `/v3/projects/b9101270b4e74050b4bc533fd069fd20/users/d3a458301d2d42dfbd77bec48df41ef0/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:34.758775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dee1594-2496-4ae9-8e4d-8daa977e5999 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9101270b4e74050b4bc533fd069fd20, user_id=d3a458301d2d42dfbd77bec48df41ef0, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:34.762146 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dee1594-2496-4ae9-8e4d-8daa977e5999 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:34.863304 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 812/1657] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:49:34 2022] PUT /identity/v3/projects/b9101270b4e74050b4bc533fd069fd20/users/d3a458301d2d42dfbd77bec48df41ef0/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:49:34.864334 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de9689-48d9-4ef4-a3e1-8487e7de0c8f admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:34.881381 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de9689-48d9-4ef4-a3e1-8487e7de0c8f admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:34.897081 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:35.006497 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1af7f7-7a80-45c7-8214-cb3de1d4579f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:35.007139 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1af7f7-7a80-45c7-8214-cb3de1d4579f admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:35.007437 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1af7f7-7a80-45c7-8214-cb3de1d4579f admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:35.007869 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1af7f7-7a80-45c7-8214-cb3de1d4579f admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:35.008656 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1af7f7-7a80-45c7-8214-cb3de1d4579f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:35.010420 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1af7f7-7a80-45c7-8214-cb3de1d4579f admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:35.033987 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dee1594-2496-4ae9-8e4d-8daa977e5999 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:35.034408 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 846/1658] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:49:35.057813 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dee1594-2496-4ae9-8e4d-8daa977e5999 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:35.069592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba86b1-e639-4f74-937a-e50d1981bc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:35.070455 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba86b1-e639-4f74-937a-e50d1981bc3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:35.070829 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba86b1-e639-4f74-937a-e50d1981bc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:35.071844 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba86b1-e639-4f74-937a-e50d1981bc3b admin admin] PATH_INFO: `/v3/projects/b9101270b4e74050b4bc533fd069fd20/users/d3a458301d2d42dfbd77bec48df41ef0/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:35.073562 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dba86b1-e639-4f74-937a-e50d1981bc3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9101270b4e74050b4bc533fd069fd20, user_id=d3a458301d2d42dfbd77bec48df41ef0, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:35.076408 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dba86b1-e639-4f74-937a-e50d1981bc3b admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:35.090417 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 813/1659] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:49:35 2022] PUT /identity/v3/projects/b9101270b4e74050b4bc533fd069fd20/users/d3a458301d2d42dfbd77bec48df41ef0/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:49:35.101343 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff03c2-44e4-4d59-aa77-056616cb1dff None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:35.101674 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff03c2-44e4-4d59-aa77-056616cb1dff None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:35.101903 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff03c2-44e4-4d59-aa77-056616cb1dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:35.154819 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-71ff03c2-44e4-4d59-aa77-056616cb1dff None None] MFA Rules not processed for user `d3a458301d2d42dfbd77bec48df41ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:35.156284 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-71ff03c2-44e4-4d59-aa77-056616cb1dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:35.284168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 847/1660] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:35 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:35.298806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba86b1-e639-4f74-937a-e50d1981bc3b admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:35.302179 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-2dba86b1-e639-4f74-937a-e50d1981bc3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:35.484792 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba86b1-e639-4f74-937a-e50d1981bc3b admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:35.494135 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df7b22-5872-48b6-9bb9-c4bf34b4ee07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:35.495045 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df7b22-5872-48b6-9bb9-c4bf34b4ee07 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:35.495991 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df7b22-5872-48b6-9bb9-c4bf34b4ee07 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:35.496439 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df7b22-5872-48b6-9bb9-c4bf34b4ee07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:35.521937 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14df7b22-5872-48b6-9bb9-c4bf34b4ee07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:35.524194 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14df7b22-5872-48b6-9bb9-c4bf34b4ee07 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:35.571420 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 814/1661] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:49:35 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:49:35.786211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff03c2-44e4-4d59-aa77-056616cb1dff None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:35.787779 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-71ff03c2-44e4-4d59-aa77-056616cb1dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:35.792462 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df7b22-5872-48b6-9bb9-c4bf34b4ee07 service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:35.794409 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-14df7b22-5872-48b6-9bb9-c4bf34b4ee07 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:35.813346 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df7b22-5872-48b6-9bb9-c4bf34b4ee07 service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:35.832975 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e867ef46-aca5-4d91-8770-eba675279cc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:35.833610 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e867ef46-aca5-4d91-8770-eba675279cc9 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:35.833887 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e867ef46-aca5-4d91-8770-eba675279cc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:35.834156 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e867ef46-aca5-4d91-8770-eba675279cc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:35.917151 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:35.975457 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e867ef46-aca5-4d91-8770-eba675279cc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:35.976914 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e867ef46-aca5-4d91-8770-eba675279cc9 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:35.997015 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 815/1662] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:49:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:49:36.043136 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff03c2-44e4-4d59-aa77-056616cb1dff None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:36.071597 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af901247-a9eb-42c9-9a39-cf814ce5c06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:36.072288 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af901247-a9eb-42c9-9a39-cf814ce5c06c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:36.074864 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af901247-a9eb-42c9-9a39-cf814ce5c06c admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:36.075162 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af901247-a9eb-42c9-9a39-cf814ce5c06c admin admin] PATH_INFO: `/v3/projects/95c05358d0df40c5971e3822352eb8b9` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:36.093916 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af901247-a9eb-42c9-9a39-cf814ce5c06c admin admin] RBAC: Authorizing `identity:delete_project(project_id=95c05358d0df40c5971e3822352eb8b9)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:36.093916 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af901247-a9eb-42c9-9a39-cf814ce5c06c admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:36.268254 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:36.287093 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-af901247-a9eb-42c9-9a39-cf814ce5c06c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95c05358d0df40c5971e3822352eb8b9', 'request_id': 'req-af901247-a9eb-42c9-9a39-cf814ce5c06c'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:36.304201 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-af901247-a9eb-42c9-9a39-cf814ce5c06c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95c05358d0df40c5971e3822352eb8b9', 'request_id': 'req-af901247-a9eb-42c9-9a39-cf814ce5c06c'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:36.414363 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-af901247-a9eb-42c9-9a39-cf814ce5c06c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95c05358d0df40c5971e3822352eb8b9', 'request_id': 'req-af901247-a9eb-42c9-9a39-cf814ce5c06c'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:36.430803 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-af901247-a9eb-42c9-9a39-cf814ce5c06c admin admin] The token cache is being invalidate because project 95c05358d0df40c5971e3822352eb8b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:49:36.431689 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-af901247-a9eb-42c9-9a39-cf814ce5c06c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:36.434442 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-af901247-a9eb-42c9-9a39-cf814ce5c06c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:36.446888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 848/1663] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:49:35 2022] DELETE /identity/v3/projects/95c05358d0df40c5971e3822352eb8b9 => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:49:36.455812 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0efb3d-d617-4f4e-b3a4-a24fcef36408 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:36.455812 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0efb3d-d617-4f4e-b3a4-a24fcef36408 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:36.455812 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0efb3d-d617-4f4e-b3a4-a24fcef36408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:36.478362 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-2f0efb3d-d617-4f4e-b3a4-a24fcef36408 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:36.480048 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-2f0efb3d-d617-4f4e-b3a4-a24fcef36408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:36.601790 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea1a39e-2cf0-4afa-af88-49965e0cc63a None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:36.602490 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea1a39e-2cf0-4afa-af88-49965e0cc63a None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:36.605825 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea1a39e-2cf0-4afa-af88-49965e0cc63a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:36.658558 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 816/1664] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:49:36.663916 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de22660-467a-46d7-b595-7d4e159c554d None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:36.663916 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de22660-467a-46d7-b595-7d4e159c554d None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:36.663916 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de22660-467a-46d7-b595-7d4e159c554d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:36.672687 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-bea1a39e-2cf0-4afa-af88-49965e0cc63a None None] MFA Rules not processed for user `ad5654cfa65f4cfa9e68832b9cd3d129`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:36.675692 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-bea1a39e-2cf0-4afa-af88-49965e0cc63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:36.715657 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-5de22660-467a-46d7-b595-7d4e159c554d None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:36.717078 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-5de22660-467a-46d7-b595-7d4e159c554d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:36.734875 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 817/1665] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:36.740265 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de22660-467a-46d7-b595-7d4e159c554d None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:36.766732 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de22660-467a-46d7-b595-7d4e159c554d None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:36.777797 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4533e93-1ced-4f9d-8649-9f18f989e203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:36.780048 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4533e93-1ced-4f9d-8649-9f18f989e203 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:36.782266 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4533e93-1ced-4f9d-8649-9f18f989e203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:36.782658 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4533e93-1ced-4f9d-8649-9f18f989e203 admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:36.785282 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4533e93-1ced-4f9d-8649-9f18f989e203 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:49:36.786384 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4533e93-1ced-4f9d-8649-9f18f989e203 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:36.793115 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4533e93-1ced-4f9d-8649-9f18f989e203 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:36.830851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 849/1666] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:36 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:36.841589 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 818/1667] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:49:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:49:36.844408 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4533e93-1ced-4f9d-8649-9f18f989e203 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:36.851201 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d3194e-d443-4605-8c9c-8106a2f5a387 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:36.851201 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d3194e-d443-4605-8c9c-8106a2f5a387 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:36.851201 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d3194e-d443-4605-8c9c-8106a2f5a387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:36.870646 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4533e93-1ced-4f9d-8649-9f18f989e203 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:36.883718 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08f1d0-eccb-46dc-9e4c-67f3a82800c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:36.883718 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08f1d0-eccb-46dc-9e4c-67f3a82800c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:36.883718 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08f1d0-eccb-46dc-9e4c-67f3a82800c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:36.883718 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08f1d0-eccb-46dc-9e4c-67f3a82800c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:36.884276 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08f1d0-eccb-46dc-9e4c-67f3a82800c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:36.885951 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08f1d0-eccb-46dc-9e4c-67f3a82800c8 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:36.890770 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-d7d3194e-d443-4605-8c9c-8106a2f5a387 None None] MFA Rules not processed for user `ad5654cfa65f4cfa9e68832b9cd3d129`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:36.898173 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-d7d3194e-d443-4605-8c9c-8106a2f5a387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:36.911011 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 850/1668] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:36 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:36.923982 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d3194e-d443-4605-8c9c-8106a2f5a387 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:36.929137 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-d7d3194e-d443-4605-8c9c-8106a2f5a387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:37.042684 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 819/1669] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:49:36 2022] POST /identity/v3/projects => generated 403 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:49:37.046826 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08f1d0-eccb-46dc-9e4c-67f3a82800c8 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:37.102424 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d3194e-d443-4605-8c9c-8106a2f5a387 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:37.206093 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08f1d0-eccb-46dc-9e4c-67f3a82800c8 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:37.221319 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ef1a6-77c9-4ffb-a7ca-125c82c02096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:37.222056 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ef1a6-77c9-4ffb-a7ca-125c82c02096 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:37.222337 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ef1a6-77c9-4ffb-a7ca-125c82c02096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:37.222671 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ef1a6-77c9-4ffb-a7ca-125c82c02096 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:37.223400 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ef1a6-77c9-4ffb-a7ca-125c82c02096 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:37.227524 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ef1a6-77c9-4ffb-a7ca-125c82c02096 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:37.287411 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfaefeb-4247-4464-97c1-71afe94b59d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:37.288514 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfaefeb-4247-4464-97c1-71afe94b59d8 service nova] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:37.289030 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfaefeb-4247-4464-97c1-71afe94b59d8 service nova] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:37.289509 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfaefeb-4247-4464-97c1-71afe94b59d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:37.360134 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 820/1670] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:49:37 2022] POST /identity/v3/users => generated 302 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:49:37.361808 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfaefeb-4247-4464-97c1-71afe94b59d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:37.367640 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ef1a6-77c9-4ffb-a7ca-125c82c02096 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:37.367774 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfaefeb-4247-4464-97c1-71afe94b59d8 service nova] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:37.381939 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ef1a6-77c9-4ffb-a7ca-125c82c02096 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:37.383113 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 851/1671] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:49:36 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:49:37.391273 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a53209-9da6-4985-af50-36338b85a6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:37.392834 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a53209-9da6-4985-af50-36338b85a6a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:37.393278 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a53209-9da6-4985-af50-36338b85a6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:37.393839 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a53209-9da6-4985-af50-36338b85a6a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:37.394622 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a53209-9da6-4985-af50-36338b85a6a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:37.397998 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a53209-9da6-4985-af50-36338b85a6a2 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:37.414662 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 821/1672] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:49:37.418255 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfaefeb-4247-4464-97c1-71afe94b59d8 service nova] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:37.440235 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfaefeb-4247-4464-97c1-71afe94b59d8 service nova] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:37.455282 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb59535-cd5e-4863-82d9-0d9a7d21a2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:37.457919 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb59535-cd5e-4863-82d9-0d9a7d21a2c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:37.458323 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb59535-cd5e-4863-82d9-0d9a7d21a2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:37.459848 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb59535-cd5e-4863-82d9-0d9a7d21a2c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:37.460665 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb59535-cd5e-4863-82d9-0d9a7d21a2c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:37.465807 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb59535-cd5e-4863-82d9-0d9a7d21a2c7 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:37.494701 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 852/1673] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:49:37.496609 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a53209-9da6-4985-af50-36338b85a6a2 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:37.512601 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a53209-9da6-4985-af50-36338b85a6a2 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:37.530329 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e683741a-64d5-4165-96bf-8846d9d6d2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:37.531026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683741a-64d5-4165-96bf-8846d9d6d2cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:37.531504 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683741a-64d5-4165-96bf-8846d9d6d2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:37.531916 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683741a-64d5-4165-96bf-8846d9d6d2cc admin admin] PATH_INFO: `/v3/projects/c88366da05654b2c8edbb4261b8a90be/users/8741999af245405dabc5f8a02032ca11/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:37.539976 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:37.569693 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e683741a-64d5-4165-96bf-8846d9d6d2cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c88366da05654b2c8edbb4261b8a90be, user_id=8741999af245405dabc5f8a02032ca11, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:37.572456 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e683741a-64d5-4165-96bf-8846d9d6d2cc admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:37.688470 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 822/1674] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:49:37 2022] PUT /identity/v3/projects/c88366da05654b2c8edbb4261b8a90be/users/8741999af245405dabc5f8a02032ca11/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:49:37.693034 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ac8f9-db3f-4479-bba5-4562553089fe None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:37.693479 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ac8f9-db3f-4479-bba5-4562553089fe None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:37.693800 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ac8f9-db3f-4479-bba5-4562553089fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:37.717573 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:37.741414 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-138ac8f9-db3f-4479-bba5-4562553089fe None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:37.746716 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-138ac8f9-db3f-4479-bba5-4562553089fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:37.926085 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 853/1675] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:37.977939 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e683741a-64d5-4165-96bf-8846d9d6d2cc admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:37.981181 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e683741a-64d5-4165-96bf-8846d9d6d2cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:38.197097 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e683741a-64d5-4165-96bf-8846d9d6d2cc admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:38.212929 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cddaaad-7178-4f78-8100-a1e8ec7a70f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:38.213458 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cddaaad-7178-4f78-8100-a1e8ec7a70f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:38.214433 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cddaaad-7178-4f78-8100-a1e8ec7a70f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:38.214716 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cddaaad-7178-4f78-8100-a1e8ec7a70f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:38.242985 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cddaaad-7178-4f78-8100-a1e8ec7a70f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:38.257580 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cddaaad-7178-4f78-8100-a1e8ec7a70f3 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:38.277899 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 823/1676] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:49:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:49:38.476669 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe0d6e3-f6f5-42d3-b46b-5fea239b0d35 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:38.477136 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe0d6e3-f6f5-42d3-b46b-5fea239b0d35 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:38.477223 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe0d6e3-f6f5-42d3-b46b-5fea239b0d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:38.557085 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-bfe0d6e3-f6f5-42d3-b46b-5fea239b0d35 None None] MFA Rules not processed for user `f938505064304ae19cd0a6df370b7721`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:38.558380 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-bfe0d6e3-f6f5-42d3-b46b-5fea239b0d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:38.610378 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bab5a3-324a-4eb4-8cb5-2a867a3b84ff None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:38.610881 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bab5a3-324a-4eb4-8cb5-2a867a3b84ff None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:38.610957 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bab5a3-324a-4eb4-8cb5-2a867a3b84ff None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:38.613452 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 824/1677] 10.222.0.33 () {54 vars in 912 bytes} [Mon Oct 3 12:49:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:49:38.619413 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce5d99-a252-45f3-b83e-8a022d4c2329 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:38.619701 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce5d99-a252-45f3-b83e-8a022d4c2329 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:38.619979 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce5d99-a252-45f3-b83e-8a022d4c2329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:38.771177 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 854/1678] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:38.786680 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf3fe4-609f-4000-9d68-51c469b3491d None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:38.787192 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf3fe4-609f-4000-9d68-51c469b3491d None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:38.793169 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf3fe4-609f-4000-9d68-51c469b3491d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:38.793676 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-b6ce5d99-a252-45f3-b83e-8a022d4c2329 None None] MFA Rules not processed for user `eb327a8a18ca414ab8b4111a4d13aad8`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:38.796537 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b6ce5d99-a252-45f3-b83e-8a022d4c2329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:38.813950 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 825/1679] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Oct 3 12:49:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:49:38.824419 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-10cf3fe4-609f-4000-9d68-51c469b3491d None None] MFA Rules not processed for user `f938505064304ae19cd0a6df370b7721`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:38.832153 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce5d99-a252-45f3-b83e-8a022d4c2329 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:38.832153 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b6ce5d99-a252-45f3-b83e-8a022d4c2329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:38.834039 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-10cf3fe4-609f-4000-9d68-51c469b3491d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:38.850998 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce5d99-a252-45f3-b83e-8a022d4c2329 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:38.856246 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 855/1680] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:38.865568 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609bddf-7c9e-4031-b0f1-b957b46c89cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:38.868949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609bddf-7c9e-4031-b0f1-b957b46c89cd service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:38.869256 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609bddf-7c9e-4031-b0f1-b957b46c89cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:38.869538 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609bddf-7c9e-4031-b0f1-b957b46c89cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:38.876136 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf3fe4-609f-4000-9d68-51c469b3491d None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:38.877676 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-10cf3fe4-609f-4000-9d68-51c469b3491d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:38.910124 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9609bddf-7c9e-4031-b0f1-b957b46c89cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:38.911976 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9609bddf-7c9e-4031-b0f1-b957b46c89cd service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:38.934180 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 826/1681] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:49:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:49:39.015356 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf3fe4-609f-4000-9d68-51c469b3491d None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:39.032149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb060dd-b531-4c35-bcad-61c6903f9fa1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:39.034685 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb060dd-b531-4c35-bcad-61c6903f9fa1 service nova] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:39.034685 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb060dd-b531-4c35-bcad-61c6903f9fa1 service nova] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:39.034685 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb060dd-b531-4c35-bcad-61c6903f9fa1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:39.052018 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb060dd-b531-4c35-bcad-61c6903f9fa1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:39.057511 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb060dd-b531-4c35-bcad-61c6903f9fa1 service nova] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:39.079011 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 856/1682] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:49:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:49:40.114036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed0ec0-5a45-466b-aac2-6017a5425e5c None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:40.114545 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed0ec0-5a45-466b-aac2-6017a5425e5c None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:40.114619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed0ec0-5a45-466b-aac2-6017a5425e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:40.147810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-56ed0ec0-5a45-466b-aac2-6017a5425e5c None None] MFA Rules not processed for user `ad5654cfa65f4cfa9e68832b9cd3d129`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:40.148387 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb060dd-b531-4c35-bcad-61c6903f9fa1 service nova] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:40.149932 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-56ed0ec0-5a45-466b-aac2-6017a5425e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:40.150039 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-9fb060dd-b531-4c35-bcad-61c6903f9fa1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:40.162200 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:40.383584 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 827/1683] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:40 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:40.385230 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47f335-a297-4814-8a62-1d63970c96a7 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:40.385853 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47f335-a297-4814-8a62-1d63970c96a7 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:40.386271 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47f335-a297-4814-8a62-1d63970c96a7 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:40.388092 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb060dd-b531-4c35-bcad-61c6903f9fa1 service nova] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:40.401125 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 828/1684] 10.222.0.33 () {54 vars in 912 bytes} [Mon Oct 3 12:49:40 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:49:40.414124 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aa5d65-39fa-432e-8dec-0bc4ca78de44 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25eef64e92e9422ea62508c3eb0a197e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:40.414779 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa5d65-39fa-432e-8dec-0bc4ca78de44 service placement] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:40.415902 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05793be-c0d0-4b53-87e8-94dcc2e7d58d None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:40.416363 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05793be-c0d0-4b53-87e8-94dcc2e7d58d None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:40.416828 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05793be-c0d0-4b53-87e8-94dcc2e7d58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:40.417468 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa5d65-39fa-432e-8dec-0bc4ca78de44 service placement] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:40.417756 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa5d65-39fa-432e-8dec-0bc4ca78de44 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:40.428882 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-54aa5d65-39fa-432e-8dec-0bc4ca78de44 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:40.462094 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-f05793be-c0d0-4b53-87e8-94dcc2e7d58d None None] MFA Rules not processed for user `eb327a8a18ca414ab8b4111a4d13aad8`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:40.465360 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-f05793be-c0d0-4b53-87e8-94dcc2e7d58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:40.488628 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aa5d65-39fa-432e-8dec-0bc4ca78de44 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:40.489857 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aa5d65-39fa-432e-8dec-0bc4ca78de44 service placement] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:40.501031 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 829/1685] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Oct 3 12:49:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:40.517801 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05793be-c0d0-4b53-87e8-94dcc2e7d58d None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:40.534725 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05793be-c0d0-4b53-87e8-94dcc2e7d58d None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:40.546842 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 857/1686] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:49:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:49:40.549056 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a90ae-3d63-41bb-b240-7a67902ab84f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:40.549916 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a90ae-3d63-41bb-b240-7a67902ab84f service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:40.550508 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a90ae-3d63-41bb-b240-7a67902ab84f service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:40.550996 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a90ae-3d63-41bb-b240-7a67902ab84f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:40.571680 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a90ae-3d63-41bb-b240-7a67902ab84f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:40.577304 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a90ae-3d63-41bb-b240-7a67902ab84f service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:40.597527 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 830/1687] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:49:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:49:40.910259 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aa5d65-39fa-432e-8dec-0bc4ca78de44 service placement] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:40.917427 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-54aa5d65-39fa-432e-8dec-0bc4ca78de44 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:40.948163 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aa5d65-39fa-432e-8dec-0bc4ca78de44 service placement] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:40.967244 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51273a3-f6a7-4bba-b11c-a9d860e15f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:40.968060 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51273a3-f6a7-4bba-b11c-a9d860e15f6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:40.968284 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51273a3-f6a7-4bba-b11c-a9d860e15f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:40.968596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51273a3-f6a7-4bba-b11c-a9d860e15f6c admin admin] PATH_INFO: `/v3/users/461cbea0d41d405ab9be8af902268ee6` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:40.972057 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51273a3-f6a7-4bba-b11c-a9d860e15f6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=461cbea0d41d405ab9be8af902268ee6)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:40.978229 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: WARNING py.warnings [None req-d51273a3-f6a7-4bba-b11c-a9d860e15f6c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 12:49:40.978229 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: warnings.warn(msg) Oct 03 12:49:40.978229 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: Oct 03 12:49:40.979525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51273a3-f6a7-4bba-b11c-a9d860e15f6c admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:41.387966 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-d51273a3-f6a7-4bba-b11c-a9d860e15f6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '461cbea0d41d405ab9be8af902268ee6'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:41.462231 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-d51273a3-f6a7-4bba-b11c-a9d860e15f6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '461cbea0d41d405ab9be8af902268ee6'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:41.528484 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:41.556055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-d51273a3-f6a7-4bba-b11c-a9d860e15f6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '461cbea0d41d405ab9be8af902268ee6'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:41.556821 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-d51273a3-f6a7-4bba-b11c-a9d860e15f6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '461cbea0d41d405ab9be8af902268ee6'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:41.689881 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-d51273a3-f6a7-4bba-b11c-a9d860e15f6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '461cbea0d41d405ab9be8af902268ee6'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:41.701049 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 858/1688] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:49:40 2022] DELETE /identity/v3/users/461cbea0d41d405ab9be8af902268ee6 => generated 0 bytes in 792 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:49:41.705418 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e6903-0518-4bb3-ba95-f00fd779dec5 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:41.706386 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e6903-0518-4bb3-ba95-f00fd779dec5 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:41.706865 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e6903-0518-4bb3-ba95-f00fd779dec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:41.766252 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-f28e6903-0518-4bb3-ba95-f00fd779dec5 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:41.770269 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-f28e6903-0518-4bb3-ba95-f00fd779dec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:41.925413 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:42.101915 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 831/1689] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:42.121389 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51273a3-f6a7-4bba-b11c-a9d860e15f6c admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:42.127101 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-d51273a3-f6a7-4bba-b11c-a9d860e15f6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:42.447678 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51273a3-f6a7-4bba-b11c-a9d860e15f6c admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:42.477077 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35894e8e-e4ce-4a36-85e4-422e8ddcf60f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:42.477848 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35894e8e-e4ce-4a36-85e4-422e8ddcf60f service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:42.478232 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35894e8e-e4ce-4a36-85e4-422e8ddcf60f service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:42.478597 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35894e8e-e4ce-4a36-85e4-422e8ddcf60f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:42.518242 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35894e8e-e4ce-4a36-85e4-422e8ddcf60f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:42.521084 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35894e8e-e4ce-4a36-85e4-422e8ddcf60f service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:42.551141 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 859/1690] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:49:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:49:43.849180 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28e6903-0518-4bb3-ba95-f00fd779dec5 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:43.850716 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-f28e6903-0518-4bb3-ba95-f00fd779dec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:43.880262 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28e6903-0518-4bb3-ba95-f00fd779dec5 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:43.886205 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae93fc2-b8b8-4f2e-ab03-affb6ab37171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:43.893362 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae93fc2-b8b8-4f2e-ab03-affb6ab37171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:43.893362 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae93fc2-b8b8-4f2e-ab03-affb6ab37171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:43.893362 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae93fc2-b8b8-4f2e-ab03-affb6ab37171 admin admin] PATH_INFO: `/v3/users/da86b18305834c19a0b7002feaad2921` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:43.893362 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae93fc2-b8b8-4f2e-ab03-affb6ab37171 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da86b18305834c19a0b7002feaad2921)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:43.895590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae93fc2-b8b8-4f2e-ab03-affb6ab37171 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:43.963995 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35894e8e-e4ce-4a36-85e4-422e8ddcf60f service neutron] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:43.965676 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-35894e8e-e4ce-4a36-85e4-422e8ddcf60f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:44.005981 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35894e8e-e4ce-4a36-85e4-422e8ddcf60f service neutron] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:44.031571 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367a279-6649-4c7f-86ff-cac30f3783d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:44.037877 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367a279-6649-4c7f-86ff-cac30f3783d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:44.038119 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367a279-6649-4c7f-86ff-cac30f3783d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:44.039077 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367a279-6649-4c7f-86ff-cac30f3783d2 admin admin] PATH_INFO: `/v3/users/49bc3ec9f78445f996378f9ddf4f928b` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:44.046037 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3367a279-6649-4c7f-86ff-cac30f3783d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49bc3ec9f78445f996378f9ddf4f928b)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:44.056206 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3367a279-6649-4c7f-86ff-cac30f3783d2 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:44.187951 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-2ae93fc2-b8b8-4f2e-ab03-affb6ab37171 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da86b18305834c19a0b7002feaad2921'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:44.228719 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-2ae93fc2-b8b8-4f2e-ab03-affb6ab37171 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da86b18305834c19a0b7002feaad2921'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:44.229649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-2ae93fc2-b8b8-4f2e-ab03-affb6ab37171 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da86b18305834c19a0b7002feaad2921'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:44.248133 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-2ae93fc2-b8b8-4f2e-ab03-affb6ab37171 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da86b18305834c19a0b7002feaad2921'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:44.321835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-2ae93fc2-b8b8-4f2e-ab03-affb6ab37171 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da86b18305834c19a0b7002feaad2921'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:44.335676 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 832/1691] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:49:43 2022] DELETE /identity/v3/users/da86b18305834c19a0b7002feaad2921 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:49:44.341683 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae93fc2-b8b8-4f2e-ab03-affb6ab37171 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:44.342409 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-3367a279-6649-4c7f-86ff-cac30f3783d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49bc3ec9f78445f996378f9ddf4f928b'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:44.344449 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-2ae93fc2-b8b8-4f2e-ab03-affb6ab37171 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:44.361775 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-3367a279-6649-4c7f-86ff-cac30f3783d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49bc3ec9f78445f996378f9ddf4f928b'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:44.413558 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-3367a279-6649-4c7f-86ff-cac30f3783d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49bc3ec9f78445f996378f9ddf4f928b'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:44.419286 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-3367a279-6649-4c7f-86ff-cac30f3783d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49bc3ec9f78445f996378f9ddf4f928b'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:44.529404 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-3367a279-6649-4c7f-86ff-cac30f3783d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49bc3ec9f78445f996378f9ddf4f928b'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:44.535736 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae93fc2-b8b8-4f2e-ab03-affb6ab37171 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:44.548212 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 860/1692] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:49:43 2022] DELETE /identity/v3/users/49bc3ec9f78445f996378f9ddf4f928b => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:49:44.551212 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4ddc8-30b6-489b-a609-c0185ccce897 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:44.554054 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4ddc8-30b6-489b-a609-c0185ccce897 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:44.554500 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4ddc8-30b6-489b-a609-c0185ccce897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:44.636524 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-9ac4ddc8-30b6-489b-a609-c0185ccce897 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:44.641843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-9ac4ddc8-30b6-489b-a609-c0185ccce897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:44.726246 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:44.763856 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:44.768966 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 861/1693] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:44.774586 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e2c87-8547-4d55-bc26-3c549100f808 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:44.784880 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e2c87-8547-4d55-bc26-3c549100f808 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:44.785184 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e2c87-8547-4d55-bc26-3c549100f808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:44.790320 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38454a9-5e89-42ac-8de7-a822084f75ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:44.791159 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38454a9-5e89-42ac-8de7-a822084f75ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:44.791619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38454a9-5e89-42ac-8de7-a822084f75ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:44.795617 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38454a9-5e89-42ac-8de7-a822084f75ef admin admin] PATH_INFO: `/v3/projects/e999f3994f1a4d4c8bf59df277dfa2e6` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:44.797525 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38454a9-5e89-42ac-8de7-a822084f75ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=e999f3994f1a4d4c8bf59df277dfa2e6)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:44.799074 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: WARNING py.warnings [None req-d38454a9-5e89-42ac-8de7-a822084f75ef admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 12:49:44.799074 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: warnings.warn(msg) Oct 03 12:49:44.799074 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: Oct 03 12:49:44.800663 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38454a9-5e89-42ac-8de7-a822084f75ef admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:44.835042 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-0d6e2c87-8547-4d55-bc26-3c549100f808 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:44.836690 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-0d6e2c87-8547-4d55-bc26-3c549100f808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:44.858432 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 862/1694] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:44.869091 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e2c87-8547-4d55-bc26-3c549100f808 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:44.870505 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-0d6e2c87-8547-4d55-bc26-3c549100f808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:45.055640 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d38454a9-5e89-42ac-8de7-a822084f75ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e999f3994f1a4d4c8bf59df277dfa2e6', 'request_id': 'req-d38454a9-5e89-42ac-8de7-a822084f75ef'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:45.073351 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d38454a9-5e89-42ac-8de7-a822084f75ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e999f3994f1a4d4c8bf59df277dfa2e6', 'request_id': 'req-d38454a9-5e89-42ac-8de7-a822084f75ef'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:45.092361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e2c87-8547-4d55-bc26-3c549100f808 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:45.198057 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d38454a9-5e89-42ac-8de7-a822084f75ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e999f3994f1a4d4c8bf59df277dfa2e6', 'request_id': 'req-d38454a9-5e89-42ac-8de7-a822084f75ef'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:45.217286 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a996711b-305d-4b94-8aba-93333fe46f3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:45.217924 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996711b-305d-4b94-8aba-93333fe46f3f service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:45.218256 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996711b-305d-4b94-8aba-93333fe46f3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:45.218588 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996711b-305d-4b94-8aba-93333fe46f3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:45.284632 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d38454a9-5e89-42ac-8de7-a822084f75ef admin admin] The token cache is being invalidate because project e999f3994f1a4d4c8bf59df277dfa2e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:49:45.284936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d38454a9-5e89-42ac-8de7-a822084f75ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:45.290714 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d38454a9-5e89-42ac-8de7-a822084f75ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:45.309754 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 833/1695] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:49:44 2022] DELETE /identity/v3/projects/e999f3994f1a4d4c8bf59df277dfa2e6 => generated 0 bytes in 959 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:49:45.314939 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38454a9-5e89-42ac-8de7-a822084f75ef admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:45.320429 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d38454a9-5e89-42ac-8de7-a822084f75ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:45.371760 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38454a9-5e89-42ac-8de7-a822084f75ef admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:45.382196 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887828b5-a605-4e06-8d1b-e774556a1539 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:45.390228 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887828b5-a605-4e06-8d1b-e774556a1539 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:45.390228 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887828b5-a605-4e06-8d1b-e774556a1539 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:45.390228 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887828b5-a605-4e06-8d1b-e774556a1539 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:45.390228 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-887828b5-a605-4e06-8d1b-e774556a1539 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:45.405273 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a996711b-305d-4b94-8aba-93333fe46f3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:45.406737 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a996711b-305d-4b94-8aba-93333fe46f3f service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:45.424884 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887828b5-a605-4e06-8d1b-e774556a1539 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:45.426953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887828b5-a605-4e06-8d1b-e774556a1539 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:45.438297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 863/1696] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:49:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:49:45.451681 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 834/1697] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:49:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:49:45.453116 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f45037-4821-4c3f-8ca5-d3048ab94068 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:45.453572 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f45037-4821-4c3f-8ca5-d3048ab94068 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:45.453681 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f45037-4821-4c3f-8ca5-d3048ab94068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:45.497324 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-07f45037-4821-4c3f-8ca5-d3048ab94068 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:45.498556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-07f45037-4821-4c3f-8ca5-d3048ab94068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:45.527703 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 835/1698] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:45.535555 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db69ff00-2b28-4996-832a-c34ae82aeec7 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:45.535555 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db69ff00-2b28-4996-832a-c34ae82aeec7 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:45.539426 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db69ff00-2b28-4996-832a-c34ae82aeec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:45.573551 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-db69ff00-2b28-4996-832a-c34ae82aeec7 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:45.575068 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-db69ff00-2b28-4996-832a-c34ae82aeec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:45.594076 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f45037-4821-4c3f-8ca5-d3048ab94068 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:45.594555 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 864/1699] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:45.608465 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f45037-4821-4c3f-8ca5-d3048ab94068 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:45.622135 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d0ae60-5763-4a96-b066-c2b1be326b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:45.622781 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d0ae60-5763-4a96-b066-c2b1be326b04 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:45.623065 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d0ae60-5763-4a96-b066-c2b1be326b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:45.623736 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d0ae60-5763-4a96-b066-c2b1be326b04 admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:45.636769 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d0ae60-5763-4a96-b066-c2b1be326b04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:49:45.636769 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d0ae60-5763-4a96-b066-c2b1be326b04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:45.636769 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d0ae60-5763-4a96-b066-c2b1be326b04 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:45.673191 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db69ff00-2b28-4996-832a-c34ae82aeec7 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:45.690705 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 836/1700] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:49:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:49:45.694353 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db69ff00-2b28-4996-832a-c34ae82aeec7 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:45.713847 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1094b34-4477-44a5-a645-717aaa54b853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:45.714742 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1094b34-4477-44a5-a645-717aaa54b853 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:45.715169 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1094b34-4477-44a5-a645-717aaa54b853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:45.715921 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1094b34-4477-44a5-a645-717aaa54b853 admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:45.717245 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1094b34-4477-44a5-a645-717aaa54b853 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:45.719164 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1094b34-4477-44a5-a645-717aaa54b853 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:45.855555 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 865/1701] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:49:45 2022] POST /identity/v3/projects => generated 365 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:49:45.859826 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d0ae60-5763-4a96-b066-c2b1be326b04 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:45.881394 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d0ae60-5763-4a96-b066-c2b1be326b04 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:45.898129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc77cdf-422c-4d30-85c0-008cc12b21ff None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:45.898129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc77cdf-422c-4d30-85c0-008cc12b21ff None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:45.898129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc77cdf-422c-4d30-85c0-008cc12b21ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:46.022990 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-9bc77cdf-422c-4d30-85c0-008cc12b21ff None None] MFA Rules not processed for user `676b2a6bbf6e4df1b47dba2a9b8e8862`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:46.024406 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-9bc77cdf-422c-4d30-85c0-008cc12b21ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:46.090133 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:46.122102 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8204823d-f43f-4151-b568-8a1c4f7cad9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:46.122839 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204823d-f43f-4151-b568-8a1c4f7cad9b admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:46.123795 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204823d-f43f-4151-b568-8a1c4f7cad9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:46.127438 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204823d-f43f-4151-b568-8a1c4f7cad9b admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:46.129479 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8204823d-f43f-4151-b568-8a1c4f7cad9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:46.135710 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8204823d-f43f-4151-b568-8a1c4f7cad9b admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:46.200529 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 866/1702] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:45 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:46.202538 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f009f2-45b8-447a-b40b-52a240be5a8a None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:46.202965 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f009f2-45b8-447a-b40b-52a240be5a8a None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:46.203417 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f009f2-45b8-447a-b40b-52a240be5a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:46.258310 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-36f009f2-45b8-447a-b40b-52a240be5a8a None None] MFA Rules not processed for user `bd24d209dc9644e8b2860a0e676acd3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:46.259708 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-36f009f2-45b8-447a-b40b-52a240be5a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:46.294400 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 837/1703] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:49:45 2022] POST /identity/v3/users => generated 283 bytes in 435 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:49:46.296394 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11efe7e-6930-4bf7-bf4c-97c6c03a76d3 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:46.296687 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11efe7e-6930-4bf7-bf4c-97c6c03a76d3 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:46.296959 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11efe7e-6930-4bf7-bf4c-97c6c03a76d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:46.384700 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-b11efe7e-6930-4bf7-bf4c-97c6c03a76d3 None None] MFA Rules not processed for user `8741999af245405dabc5f8a02032ca11`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:46.387303 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b11efe7e-6930-4bf7-bf4c-97c6c03a76d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:46.396326 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 867/1704] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:46.397239 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f009f2-45b8-447a-b40b-52a240be5a8a None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:46.401375 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-36f009f2-45b8-447a-b40b-52a240be5a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:46.535464 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:46.620761 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 838/1705] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:46 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:46.621663 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11efe7e-6930-4bf7-bf4c-97c6c03a76d3 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:46.639773 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11efe7e-6930-4bf7-bf4c-97c6c03a76d3 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:46.654991 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6bf097-e967-47d4-bbe3-75f35198e0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:46.655795 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6bf097-e967-47d4-bbe3-75f35198e0ea admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:46.656074 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6bf097-e967-47d4-bbe3-75f35198e0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:46.656342 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6bf097-e967-47d4-bbe3-75f35198e0ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:46.657101 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6bf097-e967-47d4-bbe3-75f35198e0ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:46.659988 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6bf097-e967-47d4-bbe3-75f35198e0ea admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:46.677647 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 839/1706] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:49:46 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:49:46.677647 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6bf097-e967-47d4-bbe3-75f35198e0ea admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:46.680244 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-eb6bf097-e967-47d4-bbe3-75f35198e0ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:46.690170 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f009f2-45b8-447a-b40b-52a240be5a8a None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:46.697751 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f51b6b-255f-47ca-9c60-0c4686014a1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:46.698847 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f51b6b-255f-47ca-9c60-0c4686014a1f service nova] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:46.699547 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f51b6b-255f-47ca-9c60-0c4686014a1f service nova] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:46.700224 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f51b6b-255f-47ca-9c60-0c4686014a1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:46.707649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6bf097-e967-47d4-bbe3-75f35198e0ea admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:46.713502 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262c415d-090e-484c-bfd5-c3b0b40b4567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:46.713761 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f51b6b-255f-47ca-9c60-0c4686014a1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:46.714141 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c415d-090e-484c-bfd5-c3b0b40b4567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:46.714418 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c415d-090e-484c-bfd5-c3b0b40b4567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:46.714737 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c415d-090e-484c-bfd5-c3b0b40b4567 admin admin] PATH_INFO: `/v3/projects/b1206a741c38411ab2498aec83388dfe` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:46.717180 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262c415d-090e-484c-bfd5-c3b0b40b4567 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1206a741c38411ab2498aec83388dfe)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:46.719049 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262c415d-090e-484c-bfd5-c3b0b40b4567 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:46.725806 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f51b6b-255f-47ca-9c60-0c4686014a1f service nova] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:46.759119 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 868/1707] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:49:46 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:49:46.764308 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f51b6b-255f-47ca-9c60-0c4686014a1f service nova] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:46.764308 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-11f51b6b-255f-47ca-9c60-0c4686014a1f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:46.777929 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f51b6b-255f-47ca-9c60-0c4686014a1f service nova] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:46.795724 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c074760a-2f3b-4947-8670-e6a0c087db14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:46.799158 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c074760a-2f3b-4947-8670-e6a0c087db14 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:46.799158 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c074760a-2f3b-4947-8670-e6a0c087db14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:46.799158 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c074760a-2f3b-4947-8670-e6a0c087db14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:46.800268 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c074760a-2f3b-4947-8670-e6a0c087db14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:46.803012 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c074760a-2f3b-4947-8670-e6a0c087db14 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:46.888822 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-262c415d-090e-484c-bfd5-c3b0b40b4567 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1206a741c38411ab2498aec83388dfe', 'request_id': 'req-262c415d-090e-484c-bfd5-c3b0b40b4567'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:46.919874 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 869/1708] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:49:46 2022] POST /identity/v3/projects => generated 405 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:49:46.920882 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c074760a-2f3b-4947-8670-e6a0c087db14 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:46.940330 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-262c415d-090e-484c-bfd5-c3b0b40b4567 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1206a741c38411ab2498aec83388dfe', 'request_id': 'req-262c415d-090e-484c-bfd5-c3b0b40b4567'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:46.964003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:47.001064 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-262c415d-090e-484c-bfd5-c3b0b40b4567 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1206a741c38411ab2498aec83388dfe', 'request_id': 'req-262c415d-090e-484c-bfd5-c3b0b40b4567'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:47.026091 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-262c415d-090e-484c-bfd5-c3b0b40b4567 admin admin] The token cache is being invalidate because project b1206a741c38411ab2498aec83388dfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:49:47.026385 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-262c415d-090e-484c-bfd5-c3b0b40b4567 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:47.027652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-262c415d-090e-484c-bfd5-c3b0b40b4567 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:47.028814 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 840/1709] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:49:46 2022] DELETE /identity/v3/projects/b1206a741c38411ab2498aec83388dfe => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:49:47.029603 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262c415d-090e-484c-bfd5-c3b0b40b4567 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:47.032929 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-262c415d-090e-484c-bfd5-c3b0b40b4567 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:47.095099 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c074760a-2f3b-4947-8670-e6a0c087db14 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:47.109073 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ddaed-f4e6-4297-b078-6d98d4ecc2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:47.109721 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ddaed-f4e6-4297-b078-6d98d4ecc2de admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:47.110006 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ddaed-f4e6-4297-b078-6d98d4ecc2de admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:47.110280 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ddaed-f4e6-4297-b078-6d98d4ecc2de admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:47.113022 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7ddaed-f4e6-4297-b078-6d98d4ecc2de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:47.121309 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7ddaed-f4e6-4297-b078-6d98d4ecc2de admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:47.151426 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262c415d-090e-484c-bfd5-c3b0b40b4567 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:47.152443 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 870/1710] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:49:46 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:49:47.156848 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ddaed-f4e6-4297-b078-6d98d4ecc2de admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:47.158689 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-3e7ddaed-f4e6-4297-b078-6d98d4ecc2de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:47.162831 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1f4da-e783-48b5-8624-4416f3f15c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:47.184256 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1f4da-e783-48b5-8624-4416f3f15c08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:47.184620 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1f4da-e783-48b5-8624-4416f3f15c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:47.184897 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1f4da-e783-48b5-8624-4416f3f15c08 admin admin] PATH_INFO: `/v3/projects/02b778cf9c9b40eba64364a8505a0b94` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:47.202667 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1f4da-e783-48b5-8624-4416f3f15c08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02b778cf9c9b40eba64364a8505a0b94)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:47.202667 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1f4da-e783-48b5-8624-4416f3f15c08 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:47.319465 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ddaed-f4e6-4297-b078-6d98d4ecc2de admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:47.337820 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0253233-f3f1-4841-b98d-ae783f51e454 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:47.338477 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0253233-f3f1-4841-b98d-ae783f51e454 service nova] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:47.341837 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0253233-f3f1-4841-b98d-ae783f51e454 service nova] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:47.341837 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0253233-f3f1-4841-b98d-ae783f51e454 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:47.345950 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-c0253233-f3f1-4841-b98d-ae783f51e454 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:47.430338 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:47.436502 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-3be1f4da-e783-48b5-8624-4416f3f15c08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02b778cf9c9b40eba64364a8505a0b94', 'request_id': 'req-3be1f4da-e783-48b5-8624-4416f3f15c08'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:47.459340 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-3be1f4da-e783-48b5-8624-4416f3f15c08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02b778cf9c9b40eba64364a8505a0b94', 'request_id': 'req-3be1f4da-e783-48b5-8624-4416f3f15c08'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:47.537675 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0253233-f3f1-4841-b98d-ae783f51e454 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:47.552586 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0253233-f3f1-4841-b98d-ae783f51e454 service nova] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:47.575483 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 871/1711] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:49:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:49:47.576323 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0253233-f3f1-4841-b98d-ae783f51e454 service nova] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:47.579038 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-c0253233-f3f1-4841-b98d-ae783f51e454 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:47.594137 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-3be1f4da-e783-48b5-8624-4416f3f15c08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02b778cf9c9b40eba64364a8505a0b94', 'request_id': 'req-3be1f4da-e783-48b5-8624-4416f3f15c08'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:47.631260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-3be1f4da-e783-48b5-8624-4416f3f15c08 admin admin] The token cache is being invalidate because project 02b778cf9c9b40eba64364a8505a0b94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:49:47.631606 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-3be1f4da-e783-48b5-8624-4416f3f15c08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:47.635975 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-3be1f4da-e783-48b5-8624-4416f3f15c08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:47.637051 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0253233-f3f1-4841-b98d-ae783f51e454 service nova] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:47.643140 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 841/1712] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:49:47 2022] DELETE /identity/v3/projects/02b778cf9c9b40eba64364a8505a0b94 => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:49:47.644024 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1f4da-e783-48b5-8624-4416f3f15c08 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:47.648587 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-3be1f4da-e783-48b5-8624-4416f3f15c08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:47.759630 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:47.793866 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf8f36-a323-4109-b201-fafbf5dba893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:47.794660 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf8f36-a323-4109-b201-fafbf5dba893 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:47.796404 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf8f36-a323-4109-b201-fafbf5dba893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:47.797317 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf8f36-a323-4109-b201-fafbf5dba893 admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:47.797561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf8f36-a323-4109-b201-fafbf5dba893 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:47.800263 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf8f36-a323-4109-b201-fafbf5dba893 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:47.866404 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1f4da-e783-48b5-8624-4416f3f15c08 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:47.896460 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d91ff6-96f9-41ef-8799-118be7d75a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:47.897207 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d91ff6-96f9-41ef-8799-118be7d75a45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:47.897668 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d91ff6-96f9-41ef-8799-118be7d75a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:47.898175 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d91ff6-96f9-41ef-8799-118be7d75a45 admin admin] PATH_INFO: `/v3/users/ca98ae0f1ae2429bb37c54e9c2b12de5` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:47.901088 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d91ff6-96f9-41ef-8799-118be7d75a45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca98ae0f1ae2429bb37c54e9c2b12de5)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:47.904948 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d91ff6-96f9-41ef-8799-118be7d75a45 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:47.965825 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 872/1713] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:49:47 2022] POST /identity/v3/users => generated 309 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:49:47.966698 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf8f36-a323-4109-b201-fafbf5dba893 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:47.969960 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-2abf8f36-a323-4109-b201-fafbf5dba893 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:48.022536 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf8f36-a323-4109-b201-fafbf5dba893 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:48.030644 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34860e49-43f2-4539-b095-806599581d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:48.031709 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34860e49-43f2-4539-b095-806599581d71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:48.034559 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34860e49-43f2-4539-b095-806599581d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:48.034559 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34860e49-43f2-4539-b095-806599581d71 admin admin] PATH_INFO: `/v3/projects/5cc4cb1efac849c3a9a00837f212f836/users/3d432f2f9b6246308f1ab2929d6d5f65/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:48.079762 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34860e49-43f2-4539-b095-806599581d71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cc4cb1efac849c3a9a00837f212f836, user_id=3d432f2f9b6246308f1ab2929d6d5f65, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:48.089541 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34860e49-43f2-4539-b095-806599581d71 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:48.274239 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 873/1714] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:49:47 2022] PUT /identity/v3/projects/5cc4cb1efac849c3a9a00837f212f836/users/3d432f2f9b6246308f1ab2929d6d5f65/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:49:48.276256 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac949a8-23c1-481e-a1c8-d8acbac726ae None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:48.276573 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac949a8-23c1-481e-a1c8-d8acbac726ae None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:48.276846 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac949a8-23c1-481e-a1c8-d8acbac726ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:48.290695 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c0d91ff6-96f9-41ef-8799-118be7d75a45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca98ae0f1ae2429bb37c54e9c2b12de5'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:48.328294 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c0d91ff6-96f9-41ef-8799-118be7d75a45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca98ae0f1ae2429bb37c54e9c2b12de5'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:48.331924 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c0d91ff6-96f9-41ef-8799-118be7d75a45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca98ae0f1ae2429bb37c54e9c2b12de5'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:48.349684 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c0d91ff6-96f9-41ef-8799-118be7d75a45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca98ae0f1ae2429bb37c54e9c2b12de5'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:48.400988 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-0ac949a8-23c1-481e-a1c8-d8acbac726ae None None] MFA Rules not processed for user `2e6e4387a7c24bbc82c22eefe2a3d138`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:48.404656 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-0ac949a8-23c1-481e-a1c8-d8acbac726ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:48.472403 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c0d91ff6-96f9-41ef-8799-118be7d75a45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca98ae0f1ae2429bb37c54e9c2b12de5'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:48.482015 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 842/1715] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:49:47 2022] DELETE /identity/v3/users/ca98ae0f1ae2429bb37c54e9c2b12de5 => generated 0 bytes in 838 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:49:48.484506 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d91ff6-96f9-41ef-8799-118be7d75a45 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:48.493370 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-c0d91ff6-96f9-41ef-8799-118be7d75a45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:48.596696 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 874/1716] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Oct 3 12:49:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:48.599991 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac949a8-23c1-481e-a1c8-d8acbac726ae None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:48.606565 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-0ac949a8-23c1-481e-a1c8-d8acbac726ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:48.677149 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:48.716343 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d91ff6-96f9-41ef-8799-118be7d75a45 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:48.748256 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635cec56-627a-4475-b805-4906b3a6f87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:48.748256 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635cec56-627a-4475-b805-4906b3a6f87d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:48.748893 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635cec56-627a-4475-b805-4906b3a6f87d admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:48.748893 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635cec56-627a-4475-b805-4906b3a6f87d admin admin] PATH_INFO: `/v3/users/d3a458301d2d42dfbd77bec48df41ef0` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:48.753959 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635cec56-627a-4475-b805-4906b3a6f87d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3a458301d2d42dfbd77bec48df41ef0)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:48.761117 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635cec56-627a-4475-b805-4906b3a6f87d admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:48.790589 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac949a8-23c1-481e-a1c8-d8acbac726ae None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:48.795884 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aebb77-a64e-40f1-8ba2-90af5538a0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:48.796346 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aebb77-a64e-40f1-8ba2-90af5538a0e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:48.796627 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aebb77-a64e-40f1-8ba2-90af5538a0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:48.796899 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aebb77-a64e-40f1-8ba2-90af5538a0e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:48.797565 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aebb77-a64e-40f1-8ba2-90af5538a0e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:48.801583 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aebb77-a64e-40f1-8ba2-90af5538a0e1 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:48.825314 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 875/1717] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:49:48 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:49:48.826771 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac11c3b-9bab-40f0-86dc-f9df9777b347 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:48.827115 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac11c3b-9bab-40f0-86dc-f9df9777b347 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:48.827418 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac11c3b-9bab-40f0-86dc-f9df9777b347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:48.879170 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-3ac11c3b-9bab-40f0-86dc-f9df9777b347 None None] MFA Rules not processed for user `3d432f2f9b6246308f1ab2929d6d5f65`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:48.880558 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-3ac11c3b-9bab-40f0-86dc-f9df9777b347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:48.898893 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:49.081307 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-635cec56-627a-4475-b805-4906b3a6f87d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3a458301d2d42dfbd77bec48df41ef0'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:49.098456 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-635cec56-627a-4475-b805-4906b3a6f87d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3a458301d2d42dfbd77bec48df41ef0'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:49.098894 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 876/1718] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:49.099964 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac11c3b-9bab-40f0-86dc-f9df9777b347 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:49.109156 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-635cec56-627a-4475-b805-4906b3a6f87d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3a458301d2d42dfbd77bec48df41ef0'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:49.117450 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac11c3b-9bab-40f0-86dc-f9df9777b347 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:49.119120 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-635cec56-627a-4475-b805-4906b3a6f87d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3a458301d2d42dfbd77bec48df41ef0'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:49.124935 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007c14a-154a-4f5e-a00b-e03c2f02b0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:49.125769 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007c14a-154a-4f5e-a00b-e03c2f02b0e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:49.126120 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007c14a-154a-4f5e-a00b-e03c2f02b0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:49.126992 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007c14a-154a-4f5e-a00b-e03c2f02b0e5 admin admin] PATH_INFO: `/v3/projects/1eb3e2e4a6c1421f9f5ce7fae68051ad/users/3f1a5651effd4245938c04420647adf5/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:49.144072 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c007c14a-154a-4f5e-a00b-e03c2f02b0e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1eb3e2e4a6c1421f9f5ce7fae68051ad, user_id=3f1a5651effd4245938c04420647adf5, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:49.146539 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c007c14a-154a-4f5e-a00b-e03c2f02b0e5 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:49.273053 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-635cec56-627a-4475-b805-4906b3a6f87d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3a458301d2d42dfbd77bec48df41ef0'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:49.279814 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 877/1719] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:49:49 2022] PUT /identity/v3/projects/1eb3e2e4a6c1421f9f5ce7fae68051ad/users/3f1a5651effd4245938c04420647adf5/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:49:49.284308 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007c14a-154a-4f5e-a00b-e03c2f02b0e5 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:49.285772 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 843/1720] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:49:48 2022] DELETE /identity/v3/users/d3a458301d2d42dfbd77bec48df41ef0 => generated 0 bytes in 803 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:49:49.287349 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635cec56-627a-4475-b805-4906b3a6f87d admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:49.296350 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-c007c14a-154a-4f5e-a00b-e03c2f02b0e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:49.302944 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-635cec56-627a-4475-b805-4906b3a6f87d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:49.461220 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007c14a-154a-4f5e-a00b-e03c2f02b0e5 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:49.535816 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635cec56-627a-4475-b805-4906b3a6f87d admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:49.553670 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c7140d-add7-43e6-bf6d-03691cffdb6d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b111d72a34144073917de4f566a58233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:49.554372 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7140d-add7-43e6-bf6d-03691cffdb6d service swift] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:49.558791 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7140d-add7-43e6-bf6d-03691cffdb6d service swift] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:49.558791 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7140d-add7-43e6-bf6d-03691cffdb6d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:49.558791 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-63c7140d-add7-43e6-bf6d-03691cffdb6d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:49.618203 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb79bb-88fb-466e-a3b2-66ced919959a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:49.619190 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb79bb-88fb-466e-a3b2-66ced919959a admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:49.619665 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb79bb-88fb-466e-a3b2-66ced919959a admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:49.620111 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb79bb-88fb-466e-a3b2-66ced919959a admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:49.622301 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfb79bb-88fb-466e-a3b2-66ced919959a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:49.625499 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfb79bb-88fb-466e-a3b2-66ced919959a admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:49.685090 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c7140d-add7-43e6-bf6d-03691cffdb6d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:49.692485 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c7140d-add7-43e6-bf6d-03691cffdb6d service swift] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:49.741072 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 844/1721] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Oct 3 12:49:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 12:49:49.742425 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c7140d-add7-43e6-bf6d-03691cffdb6d service swift] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:49.745093 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-63c7140d-add7-43e6-bf6d-03691cffdb6d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:49.785404 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 878/1722] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:49:49 2022] POST /identity/v3/projects => generated 363 bytes in 505 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:49:49.786573 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb79bb-88fb-466e-a3b2-66ced919959a admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:49.794566 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c7140d-add7-43e6-bf6d-03691cffdb6d service swift] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:49.998167 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:50.000362 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae05d3c-3597-4d86-ae26-32dc55082746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:50.005859 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae05d3c-3597-4d86-ae26-32dc55082746 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:50.007476 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae05d3c-3597-4d86-ae26-32dc55082746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:50.009038 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae05d3c-3597-4d86-ae26-32dc55082746 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:50.010086 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae05d3c-3597-4d86-ae26-32dc55082746 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:50.012557 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae05d3c-3597-4d86-ae26-32dc55082746 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:50.035810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 845/1723] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:49:50.039805 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae05d3c-3597-4d86-ae26-32dc55082746 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:50.055309 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae05d3c-3597-4d86-ae26-32dc55082746 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:50.063022 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c24bd9-b175-486c-af6d-abcb9983b880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:50.064077 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c24bd9-b175-486c-af6d-abcb9983b880 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:50.064980 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c24bd9-b175-486c-af6d-abcb9983b880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:50.065401 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c24bd9-b175-486c-af6d-abcb9983b880 admin admin] PATH_INFO: `/v3/projects/1eb3e2e4a6c1421f9f5ce7fae68051ad/users/3f1a5651effd4245938c04420647adf5/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:50.073572 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c24bd9-b175-486c-af6d-abcb9983b880 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1eb3e2e4a6c1421f9f5ce7fae68051ad, user_id=3f1a5651effd4245938c04420647adf5, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:50.075728 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb79bb-88fb-466e-a3b2-66ced919959a admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:50.078200 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c24bd9-b175-486c-af6d-abcb9983b880 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:50.106292 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb587c8-a87e-4f9f-ad77-7c7df87f2796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:50.109537 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb587c8-a87e-4f9f-ad77-7c7df87f2796 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:50.110311 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb587c8-a87e-4f9f-ad77-7c7df87f2796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:50.110730 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb587c8-a87e-4f9f-ad77-7c7df87f2796 admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:50.112135 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 846/1724] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:49:50 2022] PUT /identity/v3/projects/1eb3e2e4a6c1421f9f5ce7fae68051ad/users/3f1a5651effd4245938c04420647adf5/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:49:50.113270 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb587c8-a87e-4f9f-ad77-7c7df87f2796 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:50.115793 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb587c8-a87e-4f9f-ad77-7c7df87f2796 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:50.247388 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 879/1725] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:49:49 2022] POST /identity/v3/users => generated 288 bytes in 461 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:49:50.252881 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c24bd9-b175-486c-af6d-abcb9983b880 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:50.272104 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:50.558406 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c24bd9-b175-486c-af6d-abcb9983b880 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:50.576977 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305b1058-2ec3-435e-8799-86eaca932dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:50.577825 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b1058-2ec3-435e-8799-86eaca932dfc admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:50.583950 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b1058-2ec3-435e-8799-86eaca932dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:50.584421 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b1058-2ec3-435e-8799-86eaca932dfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:50.592109 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305b1058-2ec3-435e-8799-86eaca932dfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:50.598087 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305b1058-2ec3-435e-8799-86eaca932dfc admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:50.677425 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 847/1726] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:49:50 2022] GET /identity/v3/roles => generated 1420 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:49:50.681989 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb587c8-a87e-4f9f-ad77-7c7df87f2796 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:50.718753 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb587c8-a87e-4f9f-ad77-7c7df87f2796 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:50.730996 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca437d6-e99f-413d-b2f1-f26ff79d534f None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:50.731308 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca437d6-e99f-413d-b2f1-f26ff79d534f None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:50.731590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca437d6-e99f-413d-b2f1-f26ff79d534f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:50.745142 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa5130-cd24-4a3c-9d45-ec5a3bc68599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:50.745836 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa5130-cd24-4a3c-9d45-ec5a3bc68599 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:50.746116 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa5130-cd24-4a3c-9d45-ec5a3bc68599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:50.746393 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa5130-cd24-4a3c-9d45-ec5a3bc68599 admin admin] PATH_INFO: `/v3/projects/57aad6f1fc244cdcba3b5a62b904eb50/users/26e8432d0cd5489d8af46fb774e93563/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:50.779803 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-1ca437d6-e99f-413d-b2f1-f26ff79d534f None None] MFA Rules not processed for user `eb327a8a18ca414ab8b4111a4d13aad8`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:50.788320 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1ca437d6-e99f-413d-b2f1-f26ff79d534f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:50.802936 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baa5130-cd24-4a3c-9d45-ec5a3bc68599 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57aad6f1fc244cdcba3b5a62b904eb50, user_id=26e8432d0cd5489d8af46fb774e93563, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:50.822517 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baa5130-cd24-4a3c-9d45-ec5a3bc68599 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:50.927055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 880/1727] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:49:50 2022] PUT /identity/v3/projects/57aad6f1fc244cdcba3b5a62b904eb50/users/26e8432d0cd5489d8af46fb774e93563/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:49:50.931454 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa5130-cd24-4a3c-9d45-ec5a3bc68599 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:50.969804 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 848/1728] 10.222.0.46 () {58 vars in 1015 bytes} [Mon Oct 3 12:49:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:49:51.059697 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca437d6-e99f-413d-b2f1-f26ff79d534f None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:51.061410 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1ca437d6-e99f-413d-b2f1-f26ff79d534f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:51.077029 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca437d6-e99f-413d-b2f1-f26ff79d534f None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:51.108958 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa5130-cd24-4a3c-9d45-ec5a3bc68599 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:51.132421 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d27a7-b0ab-4a3f-95ea-8f9ce3a71c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:51.133071 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d27a7-b0ab-4a3f-95ea-8f9ce3a71c38 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:51.133352 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d27a7-b0ab-4a3f-95ea-8f9ce3a71c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:51.133625 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d27a7-b0ab-4a3f-95ea-8f9ce3a71c38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:51.134282 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1d27a7-b0ab-4a3f-95ea-8f9ce3a71c38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:51.144364 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1d27a7-b0ab-4a3f-95ea-8f9ce3a71c38 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:51.172919 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 881/1729] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:49:50 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 12:49:51.176119 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d27a7-b0ab-4a3f-95ea-8f9ce3a71c38 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:51.193016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f7dd48-94c7-449a-a7a3-3cafac8a0f31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:51.193809 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7dd48-94c7-449a-a7a3-3cafac8a0f31 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:51.197638 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7dd48-94c7-449a-a7a3-3cafac8a0f31 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:51.197638 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7dd48-94c7-449a-a7a3-3cafac8a0f31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:51.202280 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d27a7-b0ab-4a3f-95ea-8f9ce3a71c38 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:51.204244 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f7dd48-94c7-449a-a7a3-3cafac8a0f31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:51.207598 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f7dd48-94c7-449a-a7a3-3cafac8a0f31 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:51.214105 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf4dd2-86b7-4242-aacf-cf3b50453185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:51.215220 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf4dd2-86b7-4242-aacf-cf3b50453185 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:51.215747 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf4dd2-86b7-4242-aacf-cf3b50453185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:51.216036 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf4dd2-86b7-4242-aacf-cf3b50453185 admin admin] PATH_INFO: `/v3/projects/57aad6f1fc244cdcba3b5a62b904eb50/users/26e8432d0cd5489d8af46fb774e93563/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:51.219719 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 849/1730] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:49:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:49:51.221862 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf4dd2-86b7-4242-aacf-cf3b50453185 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57aad6f1fc244cdcba3b5a62b904eb50, user_id=26e8432d0cd5489d8af46fb774e93563, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:51.235428 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf4dd2-86b7-4242-aacf-cf3b50453185 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:51.300738 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 882/1731] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:49:51 2022] PUT /identity/v3/projects/57aad6f1fc244cdcba3b5a62b904eb50/users/26e8432d0cd5489d8af46fb774e93563/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:49:51.309410 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c244e-6e1c-4d5c-a57f-2cfe2ebb4617 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:51.309410 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c244e-6e1c-4d5c-a57f-2cfe2ebb4617 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:51.309410 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c244e-6e1c-4d5c-a57f-2cfe2ebb4617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:51.397153 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-f61c244e-6e1c-4d5c-a57f-2cfe2ebb4617 None None] MFA Rules not processed for user `26e8432d0cd5489d8af46fb774e93563`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:51.398867 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-f61c244e-6e1c-4d5c-a57f-2cfe2ebb4617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:51.706797 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 850/1732] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:51 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 405 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:51.721037 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf4dd2-86b7-4242-aacf-cf3b50453185 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:51.723663 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-49cf4dd2-86b7-4242-aacf-cf3b50453185 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:51.872110 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf4dd2-86b7-4242-aacf-cf3b50453185 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:51.900038 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1c9e4-d477-4ccf-b654-ed3381f3a65d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:51.900667 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1c9e4-d477-4ccf-b654-ed3381f3a65d service cinder] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:51.900995 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1c9e4-d477-4ccf-b654-ed3381f3a65d service cinder] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:51.901279 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1c9e4-d477-4ccf-b654-ed3381f3a65d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:51.958687 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a1c9e4-d477-4ccf-b654-ed3381f3a65d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:51.966096 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a1c9e4-d477-4ccf-b654-ed3381f3a65d service cinder] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:51.992518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 883/1733] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:49:51 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:49:52.084498 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c244e-6e1c-4d5c-a57f-2cfe2ebb4617 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:52.086033 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-f61c244e-6e1c-4d5c-a57f-2cfe2ebb4617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:52.191926 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:52.270045 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c244e-6e1c-4d5c-a57f-2cfe2ebb4617 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:52.276095 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e29427-513a-47d8-9fdd-a926d07c0745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:52.277490 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e29427-513a-47d8-9fdd-a926d07c0745 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:52.278379 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e29427-513a-47d8-9fdd-a926d07c0745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:52.278887 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e29427-513a-47d8-9fdd-a926d07c0745 admin admin] PATH_INFO: `/v3/projects/9661cd7492664918b8bd60a504c97ccd` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:52.280374 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e29427-513a-47d8-9fdd-a926d07c0745 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9661cd7492664918b8bd60a504c97ccd)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:52.282911 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e29427-513a-47d8-9fdd-a926d07c0745 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:52.452544 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e0e29427-513a-47d8-9fdd-a926d07c0745 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9661cd7492664918b8bd60a504c97ccd', 'request_id': 'req-e0e29427-513a-47d8-9fdd-a926d07c0745'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:52.464666 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e0e29427-513a-47d8-9fdd-a926d07c0745 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9661cd7492664918b8bd60a504c97ccd', 'request_id': 'req-e0e29427-513a-47d8-9fdd-a926d07c0745'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:52.582392 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1c9e4-d477-4ccf-b654-ed3381f3a65d service cinder] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:52.584392 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-02a1c9e4-d477-4ccf-b654-ed3381f3a65d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:52.592459 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e0e29427-513a-47d8-9fdd-a926d07c0745 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9661cd7492664918b8bd60a504c97ccd', 'request_id': 'req-e0e29427-513a-47d8-9fdd-a926d07c0745'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:52.614352 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1c9e4-d477-4ccf-b654-ed3381f3a65d service cinder] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:52.625454 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:52.635028 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e0e29427-513a-47d8-9fdd-a926d07c0745 admin admin] The token cache is being invalidate because project 9661cd7492664918b8bd60a504c97ccd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:49:52.638806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e0e29427-513a-47d8-9fdd-a926d07c0745 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:52.645180 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e0e29427-513a-47d8-9fdd-a926d07c0745 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:52.656846 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 851/1734] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:49:52 2022] DELETE /identity/v3/projects/9661cd7492664918b8bd60a504c97ccd => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:49:52.663444 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ada94-ccf4-4354-8152-eed590197f53 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:52.665266 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ada94-ccf4-4354-8152-eed590197f53 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:52.667587 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ada94-ccf4-4354-8152-eed590197f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:52.702887 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-cc0ada94-ccf4-4354-8152-eed590197f53 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:52.704606 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-cc0ada94-ccf4-4354-8152-eed590197f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:52.735386 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa47e5-5440-49d1-8e19-3b77d4828837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:52.744237 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa47e5-5440-49d1-8e19-3b77d4828837 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:52.744741 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa47e5-5440-49d1-8e19-3b77d4828837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:52.745282 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa47e5-5440-49d1-8e19-3b77d4828837 admin admin] PATH_INFO: `/v3/projects/b9101270b4e74050b4bc533fd069fd20` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:52.746704 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfa47e5-5440-49d1-8e19-3b77d4828837 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9101270b4e74050b4bc533fd069fd20)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:52.748950 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfa47e5-5440-49d1-8e19-3b77d4828837 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:52.821158 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 852/1735] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:52.825290 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542495d-6415-47ff-9f8e-d50ddb4f3550 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:52.825935 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542495d-6415-47ff-9f8e-d50ddb4f3550 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:52.826452 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542495d-6415-47ff-9f8e-d50ddb4f3550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:52.884321 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-1542495d-6415-47ff-9f8e-d50ddb4f3550 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:52.886731 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1542495d-6415-47ff-9f8e-d50ddb4f3550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:52.899257 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 853/1736] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:52.903925 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542495d-6415-47ff-9f8e-d50ddb4f3550 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:52.918320 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542495d-6415-47ff-9f8e-d50ddb4f3550 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:52.958913 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-bdfa47e5-5440-49d1-8e19-3b77d4828837 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9101270b4e74050b4bc533fd069fd20', 'request_id': 'req-bdfa47e5-5440-49d1-8e19-3b77d4828837'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:53.008565 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-bdfa47e5-5440-49d1-8e19-3b77d4828837 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9101270b4e74050b4bc533fd069fd20', 'request_id': 'req-bdfa47e5-5440-49d1-8e19-3b77d4828837'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:53.117631 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-bdfa47e5-5440-49d1-8e19-3b77d4828837 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9101270b4e74050b4bc533fd069fd20', 'request_id': 'req-bdfa47e5-5440-49d1-8e19-3b77d4828837'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:53.137916 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e6100-e0e2-45ee-8664-7aa208f3b536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:53.139291 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-bdfa47e5-5440-49d1-8e19-3b77d4828837 admin admin] The token cache is being invalidate because project b9101270b4e74050b4bc533fd069fd20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:49:53.139790 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-bdfa47e5-5440-49d1-8e19-3b77d4828837 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:53.141792 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e6100-e0e2-45ee-8664-7aa208f3b536 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:53.142612 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e6100-e0e2-45ee-8664-7aa208f3b536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:53.144823 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-bdfa47e5-5440-49d1-8e19-3b77d4828837 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:53.145844 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e6100-e0e2-45ee-8664-7aa208f3b536 admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:53.147492 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e6100-e0e2-45ee-8664-7aa208f3b536 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:49:53.148218 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e6100-e0e2-45ee-8664-7aa208f3b536 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:53.150523 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 884/1737] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:49:52 2022] DELETE /identity/v3/projects/b9101270b4e74050b4bc533fd069fd20 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:49:53.153490 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2bf46-837e-42ad-bce1-e7d3a6ec8591 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:53.153789 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2bf46-837e-42ad-bce1-e7d3a6ec8591 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:53.154217 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2bf46-837e-42ad-bce1-e7d3a6ec8591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:53.155178 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e6100-e0e2-45ee-8664-7aa208f3b536 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:53.196434 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-ced2bf46-837e-42ad-bce1-e7d3a6ec8591 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:53.197504 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-ced2bf46-837e-42ad-bce1-e7d3a6ec8591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:53.207130 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 854/1738] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:49:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:49:53.208396 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e6100-e0e2-45ee-8664-7aa208f3b536 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:53.212307 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-da5e6100-e0e2-45ee-8664-7aa208f3b536 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:53.221582 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 885/1739] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:53.234626 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa67db37-a9ad-494a-901c-f1bb85f274b9 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:53.235439 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa67db37-a9ad-494a-901c-f1bb85f274b9 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:53.235439 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa67db37-a9ad-494a-901c-f1bb85f274b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:53.245669 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e6100-e0e2-45ee-8664-7aa208f3b536 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:53.275373 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e52d0-bbe1-4969-9eb7-a9f63e4d2e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:53.276114 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e52d0-bbe1-4969-9eb7-a9f63e4d2e91 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:53.276419 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e52d0-bbe1-4969-9eb7-a9f63e4d2e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:53.276696 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e52d0-bbe1-4969-9eb7-a9f63e4d2e91 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:53.277394 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958e52d0-bbe1-4969-9eb7-a9f63e4d2e91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:53.286496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958e52d0-bbe1-4969-9eb7-a9f63e4d2e91 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:53.306380 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:53.324489 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-aa67db37-a9ad-494a-901c-f1bb85f274b9 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:53.326114 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-aa67db37-a9ad-494a-901c-f1bb85f274b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:53.351667 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 886/1740] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:53.359197 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa67db37-a9ad-494a-901c-f1bb85f274b9 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:53.387532 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa67db37-a9ad-494a-901c-f1bb85f274b9 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:53.401412 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b38c1-434f-478c-b740-e6952526a992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:53.402317 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b38c1-434f-478c-b740-e6952526a992 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:53.402999 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b38c1-434f-478c-b740-e6952526a992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:53.403530 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b38c1-434f-478c-b740-e6952526a992 admin admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:53.408650 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9b38c1-434f-478c-b740-e6952526a992 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:49:53.415133 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9b38c1-434f-478c-b740-e6952526a992 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:53.419449 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9b38c1-434f-478c-b740-e6952526a992 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:53.424199 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 855/1741] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:49:53 2022] POST /identity/v3/projects => generated 357 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:49:53.431826 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e52d0-bbe1-4969-9eb7-a9f63e4d2e91 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:53.480258 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 887/1742] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:49:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:49:53.484583 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b38c1-434f-478c-b740-e6952526a992 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:53.508766 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b38c1-434f-478c-b740-e6952526a992 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:53.618343 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e52d0-bbe1-4969-9eb7-a9f63e4d2e91 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:53.640372 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0f04fb-8d30-4ee4-92d2-fe3ec6471904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:53.641814 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f04fb-8d30-4ee4-92d2-fe3ec6471904 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:53.642930 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f04fb-8d30-4ee4-92d2-fe3ec6471904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:53.644318 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f04fb-8d30-4ee4-92d2-fe3ec6471904 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:53.645691 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0f04fb-8d30-4ee4-92d2-fe3ec6471904 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:53.649672 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0f04fb-8d30-4ee4-92d2-fe3ec6471904 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:53.669350 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d70a7e3-09f7-49a2-ac42-197d49b8ad1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:53.669961 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70a7e3-09f7-49a2-ac42-197d49b8ad1f admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:53.670245 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70a7e3-09f7-49a2-ac42-197d49b8ad1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:53.670553 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70a7e3-09f7-49a2-ac42-197d49b8ad1f admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:53.671200 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d70a7e3-09f7-49a2-ac42-197d49b8ad1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:53.680408 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d70a7e3-09f7-49a2-ac42-197d49b8ad1f admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:53.814810 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:53.825250 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 856/1743] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:49:53 2022] POST /identity/v3/users => generated 279 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:49:53.825250 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0f04fb-8d30-4ee4-92d2-fe3ec6471904 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:53.825878 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 888/1744] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:49:53 2022] POST /identity/v3/projects => generated 387 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:49:53.827584 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d70a7e3-09f7-49a2-ac42-197d49b8ad1f admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:53.830626 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-8c0f04fb-8d30-4ee4-92d2-fe3ec6471904 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:53.854165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0f04fb-8d30-4ee4-92d2-fe3ec6471904 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:54.007078 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd90041-79e9-4ff8-b03a-53d23e2d188c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:54.007730 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd90041-79e9-4ff8-b03a-53d23e2d188c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:54.008196 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd90041-79e9-4ff8-b03a-53d23e2d188c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:54.008616 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd90041-79e9-4ff8-b03a-53d23e2d188c admin admin] PATH_INFO: `/v3/users/3d432f2f9b6246308f1ab2929d6d5f65` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:54.009780 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd90041-79e9-4ff8-b03a-53d23e2d188c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d432f2f9b6246308f1ab2929d6d5f65)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:54.013826 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd90041-79e9-4ff8-b03a-53d23e2d188c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:54.030149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d70a7e3-09f7-49a2-ac42-197d49b8ad1f admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:54.053627 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f0239-4b2d-4b05-b1f8-aa5f445e6f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:54.057437 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f0239-4b2d-4b05-b1f8-aa5f445e6f53 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:54.058147 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f0239-4b2d-4b05-b1f8-aa5f445e6f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:54.062394 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f0239-4b2d-4b05-b1f8-aa5f445e6f53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:54.062394 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f0239-4b2d-4b05-b1f8-aa5f445e6f53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:54.072996 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f0239-4b2d-4b05-b1f8-aa5f445e6f53 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:54.109818 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 889/1745] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:49:53 2022] GET /identity/v3/roles => generated 1420 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:49:54.112594 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f0239-4b2d-4b05-b1f8-aa5f445e6f53 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:54.135714 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f0239-4b2d-4b05-b1f8-aa5f445e6f53 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:54.150800 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ba5bf-d424-48fd-a752-9d9d7de2de83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:54.150967 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ba5bf-d424-48fd-a752-9d9d7de2de83 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:54.151257 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ba5bf-d424-48fd-a752-9d9d7de2de83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:54.152273 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ba5bf-d424-48fd-a752-9d9d7de2de83 admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:54.153672 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35ba5bf-d424-48fd-a752-9d9d7de2de83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:54.159511 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35ba5bf-d424-48fd-a752-9d9d7de2de83 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:54.302440 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 890/1746] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:49:54 2022] POST /identity/v3/users => generated 294 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:49:54.303442 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ba5bf-d424-48fd-a752-9d9d7de2de83 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:54.334157 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ba5bf-d424-48fd-a752-9d9d7de2de83 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:54.340417 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7dd90041-79e9-4ff8-b03a-53d23e2d188c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d432f2f9b6246308f1ab2929d6d5f65'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:54.340732 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4a71ec-e99b-49b3-bfd5-b616b9fcc174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:54.353082 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a71ec-e99b-49b3-bfd5-b616b9fcc174 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:54.353394 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a71ec-e99b-49b3-bfd5-b616b9fcc174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:54.354387 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a71ec-e99b-49b3-bfd5-b616b9fcc174 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:54.355482 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4a71ec-e99b-49b3-bfd5-b616b9fcc174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:54.357604 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4a71ec-e99b-49b3-bfd5-b616b9fcc174 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:54.368912 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:54.390041 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7dd90041-79e9-4ff8-b03a-53d23e2d188c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d432f2f9b6246308f1ab2929d6d5f65'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:54.392765 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7dd90041-79e9-4ff8-b03a-53d23e2d188c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d432f2f9b6246308f1ab2929d6d5f65'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:54.404404 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 891/1747] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:49:54 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:49:54.409312 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4a71ec-e99b-49b3-bfd5-b616b9fcc174 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:54.416923 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-1f4a71ec-e99b-49b3-bfd5-b616b9fcc174 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:54.418002 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7dd90041-79e9-4ff8-b03a-53d23e2d188c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d432f2f9b6246308f1ab2929d6d5f65'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:54.473602 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4a71ec-e99b-49b3-bfd5-b616b9fcc174 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:54.482507 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f26453-bd4c-401d-aa5a-17948acf6582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:54.485422 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f26453-bd4c-401d-aa5a-17948acf6582 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:54.485422 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f26453-bd4c-401d-aa5a-17948acf6582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:54.485422 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f26453-bd4c-401d-aa5a-17948acf6582 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:54.485422 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f26453-bd4c-401d-aa5a-17948acf6582 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:54.488370 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f26453-bd4c-401d-aa5a-17948acf6582 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:54.531042 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 892/1748] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:49:54 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:49:54.534207 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f26453-bd4c-401d-aa5a-17948acf6582 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:54.536068 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-f3f26453-bd4c-401d-aa5a-17948acf6582 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:54.541156 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7dd90041-79e9-4ff8-b03a-53d23e2d188c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d432f2f9b6246308f1ab2929d6d5f65'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:54.571707 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 857/1749] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:49:53 2022] DELETE /identity/v3/users/3d432f2f9b6246308f1ab2929d6d5f65 => generated 0 bytes in 744 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:49:54.571707 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd90041-79e9-4ff8-b03a-53d23e2d188c admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:54.633041 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f26453-bd4c-401d-aa5a-17948acf6582 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:54.802843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf3c488-0714-4847-84fc-51f18a4fe21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:54.804527 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf3c488-0714-4847-84fc-51f18a4fe21d admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:54.804869 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd90041-79e9-4ff8-b03a-53d23e2d188c admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:54.805362 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf3c488-0714-4847-84fc-51f18a4fe21d admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:54.805937 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf3c488-0714-4847-84fc-51f18a4fe21d admin admin] PATH_INFO: `/v3/projects/952439e0f1014185b52fddea6c65fee3/users/417b5a24768a4a8991344408f6d003d4/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:54.818270 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230d8c5-3a95-40d3-85b2-3377f22b80e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:54.818964 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230d8c5-3a95-40d3-85b2-3377f22b80e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:54.819244 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230d8c5-3a95-40d3-85b2-3377f22b80e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:54.819632 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230d8c5-3a95-40d3-85b2-3377f22b80e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:54.820390 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6230d8c5-3a95-40d3-85b2-3377f22b80e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:54.822130 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6230d8c5-3a95-40d3-85b2-3377f22b80e3 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:54.844140 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf3c488-0714-4847-84fc-51f18a4fe21d admin admin] RBAC: Authorizing `identity:create_grant(project_id=952439e0f1014185b52fddea6c65fee3, user_id=417b5a24768a4a8991344408f6d003d4, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:54.849371 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf3c488-0714-4847-84fc-51f18a4fe21d admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:54.872210 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 858/1750] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:49:54 2022] GET /identity/v3/roles => generated 1420 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:49:54.873694 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c6d35-2031-469f-989b-cf7974ffebf0 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:54.873982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c6d35-2031-469f-989b-cf7974ffebf0 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:54.874270 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c6d35-2031-469f-989b-cf7974ffebf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:54.900933 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-fc0c6d35-2031-469f-989b-cf7974ffebf0 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:54.902517 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-fc0c6d35-2031-469f-989b-cf7974ffebf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:54.916449 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 859/1751] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:54.916449 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0c6d35-2031-469f-989b-cf7974ffebf0 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:54.930686 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0c6d35-2031-469f-989b-cf7974ffebf0 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:54.945334 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd3015c-b9de-4f98-9b89-672b7f273e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:54.945334 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3015c-b9de-4f98-9b89-672b7f273e19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:54.945334 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3015c-b9de-4f98-9b89-672b7f273e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:54.945334 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3015c-b9de-4f98-9b89-672b7f273e19 admin admin] PATH_INFO: `/v3/projects/1c505a5b81c8490c899486f892830a8a/users/ceb8a4e571544a93b2b17aa2bf594657/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:54.979760 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 893/1752] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:49:54 2022] PUT /identity/v3/projects/952439e0f1014185b52fddea6c65fee3/users/417b5a24768a4a8991344408f6d003d4/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:49:54.987967 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd3015c-b9de-4f98-9b89-672b7f273e19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c505a5b81c8490c899486f892830a8a, user_id=ceb8a4e571544a93b2b17aa2bf594657, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:54.991676 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a5e85-9cd5-4cde-95e1-03713086c85d None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:54.991969 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a5e85-9cd5-4cde-95e1-03713086c85d None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:54.992247 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a5e85-9cd5-4cde-95e1-03713086c85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:54.994017 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd3015c-b9de-4f98-9b89-672b7f273e19 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:55.068502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-867a5e85-9cd5-4cde-95e1-03713086c85d None None] MFA Rules not processed for user `417b5a24768a4a8991344408f6d003d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:55.070058 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-867a5e85-9cd5-4cde-95e1-03713086c85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:55.136481 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 860/1753] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:49:54 2022] PUT /identity/v3/projects/1c505a5b81c8490c899486f892830a8a/users/ceb8a4e571544a93b2b17aa2bf594657/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:49:55.137452 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd3015c-b9de-4f98-9b89-672b7f273e19 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:55.145145 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-8cd3015c-b9de-4f98-9b89-672b7f273e19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:55.234376 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 894/1754] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:54 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:49:55.237700 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395b4ff-1c50-4f9a-b7b8-e957860ba49a None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:55.248909 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395b4ff-1c50-4f9a-b7b8-e957860ba49a None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:55.251013 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395b4ff-1c50-4f9a-b7b8-e957860ba49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:55.313715 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd3015c-b9de-4f98-9b89-672b7f273e19 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:55.324080 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3528bdb1-e51f-448d-8336-467a96dd4461 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:55.325013 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528bdb1-e51f-448d-8336-467a96dd4461 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:55.325451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528bdb1-e51f-448d-8336-467a96dd4461 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:55.325887 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528bdb1-e51f-448d-8336-467a96dd4461 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:55.393447 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-3395b4ff-1c50-4f9a-b7b8-e957860ba49a None None] MFA Rules not processed for user `ceb8a4e571544a93b2b17aa2bf594657`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:55.413604 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-3395b4ff-1c50-4f9a-b7b8-e957860ba49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:55.461125 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:55.515562 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3528bdb1-e51f-448d-8336-467a96dd4461 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:55.517384 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3528bdb1-e51f-448d-8336-467a96dd4461 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:55.539553 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 861/1755] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:49:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:49:55.541783 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3528bdb1-e51f-448d-8336-467a96dd4461 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:55.555837 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3528bdb1-e51f-448d-8336-467a96dd4461 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:55.560956 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f78a8-d612-47f4-a747-81dbd2656e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:55.561887 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f78a8-d612-47f4-a747-81dbd2656e93 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:55.562419 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f78a8-d612-47f4-a747-81dbd2656e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:55.562867 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f78a8-d612-47f4-a747-81dbd2656e93 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:55.563746 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238f78a8-d612-47f4-a747-81dbd2656e93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:55.566048 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238f78a8-d612-47f4-a747-81dbd2656e93 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:55.655104 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 895/1756] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:55 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 420 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:55.660722 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 862/1757] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:49:55 2022] POST /identity/v3/projects => generated 355 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:49:55.661926 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395b4ff-1c50-4f9a-b7b8-e957860ba49a None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:55.664591 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f78a8-d612-47f4-a747-81dbd2656e93 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:55.689875 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395b4ff-1c50-4f9a-b7b8-e957860ba49a None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:55.829702 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa948b-741b-44dc-bd82-c6fc7ac2c3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:55.829702 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa948b-741b-44dc-bd82-c6fc7ac2c3c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:55.829702 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa948b-741b-44dc-bd82-c6fc7ac2c3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:55.829702 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa948b-741b-44dc-bd82-c6fc7ac2c3c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:55.830201 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baa948b-741b-44dc-bd82-c6fc7ac2c3c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:55.832930 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baa948b-741b-44dc-bd82-c6fc7ac2c3c0 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:55.855056 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f78a8-d612-47f4-a747-81dbd2656e93 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:55.876908 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e76326e-6998-43e9-b82f-630a7cfc34ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:55.878400 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76326e-6998-43e9-b82f-630a7cfc34ec admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:55.879064 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76326e-6998-43e9-b82f-630a7cfc34ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:55.879545 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76326e-6998-43e9-b82f-630a7cfc34ec admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:55.880439 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e76326e-6998-43e9-b82f-630a7cfc34ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:55.883998 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e76326e-6998-43e9-b82f-630a7cfc34ec admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:55.976829 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 896/1758] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:49:55 2022] POST /identity/v3/projects => generated 387 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:49:55.977847 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa948b-741b-44dc-bd82-c6fc7ac2c3c0 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:55.999112 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 863/1759] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:49:55 2022] POST /identity/v3/users => generated 284 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:49:56.000267 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e76326e-6998-43e9-b82f-630a7cfc34ec admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:56.024842 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e76326e-6998-43e9-b82f-630a7cfc34ec admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:56.166914 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11084f-666c-4cef-943e-c051dec76b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:56.167649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11084f-666c-4cef-943e-c051dec76b1f admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:56.167928 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11084f-666c-4cef-943e-c051dec76b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:56.168198 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11084f-666c-4cef-943e-c051dec76b1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:56.168850 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f11084f-666c-4cef-943e-c051dec76b1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:56.170515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f11084f-666c-4cef-943e-c051dec76b1f admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:56.183060 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa948b-741b-44dc-bd82-c6fc7ac2c3c0 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:56.194541 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae5e6c1-e2a0-4cf7-8e0e-ec2c67664184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:56.194541 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5e6c1-e2a0-4cf7-8e0e-ec2c67664184 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:56.194541 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5e6c1-e2a0-4cf7-8e0e-ec2c67664184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:56.194541 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5e6c1-e2a0-4cf7-8e0e-ec2c67664184 admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:56.194888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae5e6c1-e2a0-4cf7-8e0e-ec2c67664184 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:56.195925 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 864/1760] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:49:55 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:49:56.196549 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae5e6c1-e2a0-4cf7-8e0e-ec2c67664184 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:56.196835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11084f-666c-4cef-943e-c051dec76b1f admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:56.217078 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11084f-666c-4cef-943e-c051dec76b1f admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:56.240331 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d643ef-f082-4d5d-9448-49f2a4a25428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:56.246946 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d643ef-f082-4d5d-9448-49f2a4a25428 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:56.246946 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d643ef-f082-4d5d-9448-49f2a4a25428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:56.246946 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d643ef-f082-4d5d-9448-49f2a4a25428 admin admin] PATH_INFO: `/v3/projects/d848e6d05516445795632b462af64f4c/users/7083bc5384ff4d7c996f95c139f23e1d/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:56.285009 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d643ef-f082-4d5d-9448-49f2a4a25428 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d848e6d05516445795632b462af64f4c, user_id=7083bc5384ff4d7c996f95c139f23e1d, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:56.287968 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d643ef-f082-4d5d-9448-49f2a4a25428 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:56.307368 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 897/1761] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:49:55 2022] POST /identity/v3/users => generated 300 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:49:56.309891 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae5e6c1-e2a0-4cf7-8e0e-ec2c67664184 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:56.338412 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae5e6c1-e2a0-4cf7-8e0e-ec2c67664184 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:56.355851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9913614-cbf9-4068-b9de-a2a5c88d0503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:56.363935 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9913614-cbf9-4068-b9de-a2a5c88d0503 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:56.363935 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9913614-cbf9-4068-b9de-a2a5c88d0503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:56.363935 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9913614-cbf9-4068-b9de-a2a5c88d0503 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:56.364915 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 865/1762] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:49:56 2022] PUT /identity/v3/projects/d848e6d05516445795632b462af64f4c/users/7083bc5384ff4d7c996f95c139f23e1d/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:49:56.364915 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d643ef-f082-4d5d-9448-49f2a4a25428 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:56.365084 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9913614-cbf9-4068-b9de-a2a5c88d0503 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:56.366126 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9913614-cbf9-4068-b9de-a2a5c88d0503 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:56.395386 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 898/1763] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:49:56.398953 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9913614-cbf9-4068-b9de-a2a5c88d0503 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:56.413237 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9913614-cbf9-4068-b9de-a2a5c88d0503 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:56.547104 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d643ef-f082-4d5d-9448-49f2a4a25428 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:56.551259 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72b2e5-8019-466c-bd86-5c2bbd110eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:56.551977 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72b2e5-8019-466c-bd86-5c2bbd110eb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:56.552263 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72b2e5-8019-466c-bd86-5c2bbd110eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:56.552565 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72b2e5-8019-466c-bd86-5c2bbd110eb4 admin admin] PATH_INFO: `/v3/projects/4ed0f7ec4435426eafce871fe179dc7a/users/9c44405819d94af3968bdd99921df3c7/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:56.555072 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92f29c5-db18-431a-9f1d-1f07fe6e0379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:56.555924 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f29c5-db18-431a-9f1d-1f07fe6e0379 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:56.556360 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f29c5-db18-431a-9f1d-1f07fe6e0379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:56.556768 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f29c5-db18-431a-9f1d-1f07fe6e0379 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:56.557631 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92f29c5-db18-431a-9f1d-1f07fe6e0379 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:56.559472 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92f29c5-db18-431a-9f1d-1f07fe6e0379 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:56.586861 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa72b2e5-8019-466c-bd86-5c2bbd110eb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ed0f7ec4435426eafce871fe179dc7a, user_id=9c44405819d94af3968bdd99921df3c7, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:56.590319 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa72b2e5-8019-466c-bd86-5c2bbd110eb4 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:56.593094 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 866/1764] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:49:56.597219 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f1baf-4aed-4609-bc5b-468ae235c7db None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:56.598141 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f1baf-4aed-4609-bc5b-468ae235c7db None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:56.598896 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f1baf-4aed-4609-bc5b-468ae235c7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:56.663793 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-484f1baf-4aed-4609-bc5b-468ae235c7db None None] MFA Rules not processed for user `3f1a5651effd4245938c04420647adf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:56.665325 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-484f1baf-4aed-4609-bc5b-468ae235c7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:56.723263 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 899/1765] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:49:56 2022] PUT /identity/v3/projects/4ed0f7ec4435426eafce871fe179dc7a/users/9c44405819d94af3968bdd99921df3c7/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:49:56.724356 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72b2e5-8019-466c-bd86-5c2bbd110eb4 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:56.817192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:56.864011 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 867/1766] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:56 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:56.868053 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f1baf-4aed-4609-bc5b-468ae235c7db None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:56.885015 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f1baf-4aed-4609-bc5b-468ae235c7db None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:56.955297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72b2e5-8019-466c-bd86-5c2bbd110eb4 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:56.964003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ae87c-6592-4a58-9c7b-4ef69c6e6b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:56.964681 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ae87c-6592-4a58-9c7b-4ef69c6e6b11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:56.964961 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ae87c-6592-4a58-9c7b-4ef69c6e6b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:56.965238 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ae87c-6592-4a58-9c7b-4ef69c6e6b11 admin admin] PATH_INFO: `/v3/projects/d848e6d05516445795632b462af64f4c/users/7083bc5384ff4d7c996f95c139f23e1d/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:56.969301 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ae87c-6592-4a58-9c7b-4ef69c6e6b11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d848e6d05516445795632b462af64f4c, user_id=7083bc5384ff4d7c996f95c139f23e1d, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:56.972020 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ae87c-6592-4a58-9c7b-4ef69c6e6b11 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:57.006589 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 900/1767] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:49:56 2022] PUT /identity/v3/projects/d848e6d05516445795632b462af64f4c/users/7083bc5384ff4d7c996f95c139f23e1d/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:49:57.017691 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e4b52-b2d8-4d0a-a8d7-62381ebcb88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:57.018678 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e4b52-b2d8-4d0a-a8d7-62381ebcb88f admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:57.019551 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e4b52-b2d8-4d0a-a8d7-62381ebcb88f admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:57.021457 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778acbb5-da7a-4c73-be3d-34d8fcfc79cc None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:57.027084 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778acbb5-da7a-4c73-be3d-34d8fcfc79cc None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:57.027084 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778acbb5-da7a-4c73-be3d-34d8fcfc79cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:57.027663 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e4b52-b2d8-4d0a-a8d7-62381ebcb88f admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:57.035340 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380e4b52-b2d8-4d0a-a8d7-62381ebcb88f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:57.039953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380e4b52-b2d8-4d0a-a8d7-62381ebcb88f admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:57.050183 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:57.065184 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 868/1768] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:49:57.068455 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e4b52-b2d8-4d0a-a8d7-62381ebcb88f admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:57.089093 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-778acbb5-da7a-4c73-be3d-34d8fcfc79cc None None] MFA Rules not processed for user `7083bc5384ff4d7c996f95c139f23e1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:57.093429 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-778acbb5-da7a-4c73-be3d-34d8fcfc79cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:57.096428 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e4b52-b2d8-4d0a-a8d7-62381ebcb88f admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:57.257471 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996d73a8-4497-4ca3-9e8a-d57f656ff97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:57.258197 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d73a8-4497-4ca3-9e8a-d57f656ff97a admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:57.258563 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d73a8-4497-4ca3-9e8a-d57f656ff97a admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:57.258888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d73a8-4497-4ca3-9e8a-d57f656ff97a admin admin] PATH_INFO: `/v3/projects/4ed0f7ec4435426eafce871fe179dc7a/users/9c44405819d94af3968bdd99921df3c7/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:57.264997 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996d73a8-4497-4ca3-9e8a-d57f656ff97a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ed0f7ec4435426eafce871fe179dc7a, user_id=9c44405819d94af3968bdd99921df3c7, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:57.271800 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996d73a8-4497-4ca3-9e8a-d57f656ff97a admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:57.285403 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 869/1769] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:49:57 2022] PUT /identity/v3/projects/4ed0f7ec4435426eafce871fe179dc7a/users/9c44405819d94af3968bdd99921df3c7/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:49:57.287357 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 901/1770] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:57 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:57.298099 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdabb439-4fef-43d3-abb7-91ff15190d5b None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:57.300243 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdabb439-4fef-43d3-abb7-91ff15190d5b None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:57.301075 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdabb439-4fef-43d3-abb7-91ff15190d5b None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:57.303585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 870/1771] 10.222.0.33 () {54 vars in 921 bytes} [Mon Oct 3 12:49:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:49:57.304909 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc17fe0-558d-42b9-b052-ff00da85bbd6 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:57.305226 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc17fe0-558d-42b9-b052-ff00da85bbd6 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:57.305562 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc17fe0-558d-42b9-b052-ff00da85bbd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:57.312599 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdabb439-4fef-43d3-abb7-91ff15190d5b None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:57.315272 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-bdabb439-4fef-43d3-abb7-91ff15190d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:57.385987 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-bcc17fe0-558d-42b9-b052-ff00da85bbd6 None None] MFA Rules not processed for user `9c44405819d94af3968bdd99921df3c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:57.387637 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-bcc17fe0-558d-42b9-b052-ff00da85bbd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:57.480127 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdabb439-4fef-43d3-abb7-91ff15190d5b None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:57.494074 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b757b-4fed-41c3-a47b-af255b1bd6c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:57.495646 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b757b-4fed-41c3-a47b-af255b1bd6c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:57.496438 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b757b-4fed-41c3-a47b-af255b1bd6c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:57.502036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b757b-4fed-41c3-a47b-af255b1bd6c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:57.621422 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:57.626534 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 902/1772] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:57 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:49:57.627952 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc17fe0-558d-42b9-b052-ff00da85bbd6 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:57.630039 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-bcc17fe0-558d-42b9-b052-ff00da85bbd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:57.647615 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc17fe0-558d-42b9-b052-ff00da85bbd6 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:57.667527 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ed36a-d3e5-496c-98e9-6964cbe0557f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:57.670337 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ed36a-d3e5-496c-98e9-6964cbe0557f service cinder] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:57.672277 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ed36a-d3e5-496c-98e9-6964cbe0557f service cinder] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:57.672578 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ed36a-d3e5-496c-98e9-6964cbe0557f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:57.707360 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3ed36a-d3e5-496c-98e9-6964cbe0557f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:57.707360 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3ed36a-d3e5-496c-98e9-6964cbe0557f service cinder] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:57.718736 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b757b-4fed-41c3-a47b-af255b1bd6c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:57.721915 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b757b-4fed-41c3-a47b-af255b1bd6c2 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:57.747974 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 871/1773] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:49:57 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:49:57.749513 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c09e4-da7b-49c7-8bdc-841d9946c3a5 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:57.749805 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c09e4-da7b-49c7-8bdc-841d9946c3a5 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:57.750210 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c09e4-da7b-49c7-8bdc-841d9946c3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:57.756981 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:57.763660 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 903/1774] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:49:57 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:49:57.804582 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-1e8c09e4-da7b-49c7-8bdc-841d9946c3a5 None None] MFA Rules not processed for user `990f5837994f4a088c656c9b33e9a0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:57.805779 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1e8c09e4-da7b-49c7-8bdc-841d9946c3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:57.850333 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ed36a-d3e5-496c-98e9-6964cbe0557f service cinder] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:58.031387 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 872/1775] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Oct 3 12:49:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:58.032594 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ed36a-d3e5-496c-98e9-6964cbe0557f service cinder] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:58.034342 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a54d61-b2fa-4b76-8f30-9dee6ba9d195 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:58.034988 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a54d61-b2fa-4b76-8f30-9dee6ba9d195 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:58.035524 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a54d61-b2fa-4b76-8f30-9dee6ba9d195 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:58.039956 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f4833-4385-47d7-a4c1-0c495c0ad19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:58.041028 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f4833-4385-47d7-a4c1-0c495c0ad19b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:58.041527 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f4833-4385-47d7-a4c1-0c495c0ad19b admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:58.042074 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f4833-4385-47d7-a4c1-0c495c0ad19b admin admin] PATH_INFO: `/v3/users/ceb8a4e571544a93b2b17aa2bf594657` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:58.043837 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03f4833-4385-47d7-a4c1-0c495c0ad19b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ceb8a4e571544a93b2b17aa2bf594657)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:58.046644 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03f4833-4385-47d7-a4c1-0c495c0ad19b admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:58.092318 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 873/1776] 10.222.0.33 () {54 vars in 921 bytes} [Mon Oct 3 12:49:58 2022] GET /identity => generated 270 bytes in 60 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:49:58.093508 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a54d61-b2fa-4b76-8f30-9dee6ba9d195 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:58.107837 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a54d61-b2fa-4b76-8f30-9dee6ba9d195 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:58.123797 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de5c9dd-56c7-40b5-affc-b952368478e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:58.124682 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5c9dd-56c7-40b5-affc-b952368478e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:58.125010 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5c9dd-56c7-40b5-affc-b952368478e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:58.125554 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5c9dd-56c7-40b5-affc-b952368478e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:58.254873 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-a03f4833-4385-47d7-a4c1-0c495c0ad19b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ceb8a4e571544a93b2b17aa2bf594657'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:58.273131 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-a03f4833-4385-47d7-a4c1-0c495c0ad19b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ceb8a4e571544a93b2b17aa2bf594657'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:58.283601 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de5c9dd-56c7-40b5-affc-b952368478e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:58.289903 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de5c9dd-56c7-40b5-affc-b952368478e1 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:58.301395 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-a03f4833-4385-47d7-a4c1-0c495c0ad19b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ceb8a4e571544a93b2b17aa2bf594657'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:58.303296 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-a03f4833-4385-47d7-a4c1-0c495c0ad19b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ceb8a4e571544a93b2b17aa2bf594657'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:58.326114 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 874/1777] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:49:58 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:49:58.327918 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de5c9dd-56c7-40b5-affc-b952368478e1 service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:58.329456 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-0de5c9dd-56c7-40b5-affc-b952368478e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:58.376647 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de5c9dd-56c7-40b5-affc-b952368478e1 service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:58.386316 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-a03f4833-4385-47d7-a4c1-0c495c0ad19b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ceb8a4e571544a93b2b17aa2bf594657'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:58.403703 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 904/1778] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:49:57 2022] DELETE /identity/v3/users/ceb8a4e571544a93b2b17aa2bf594657 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:49:58.403703 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61802c48-bc17-4ad1-903e-1b51d5db9bc3 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:58.403703 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61802c48-bc17-4ad1-903e-1b51d5db9bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:58.403703 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61802c48-bc17-4ad1-903e-1b51d5db9bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:58.416765 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc7135d-91e8-4785-b922-d1639afed4cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:58.418026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7135d-91e8-4785-b922-d1639afed4cc service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:58.418401 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7135d-91e8-4785-b922-d1639afed4cc service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:58.418726 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7135d-91e8-4785-b922-d1639afed4cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:58.426110 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-0cc7135d-91e8-4785-b922-d1639afed4cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:58.482541 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc7135d-91e8-4785-b922-d1639afed4cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:58.486067 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-61802c48-bc17-4ad1-903e-1b51d5db9bc3 None None] MFA Rules not processed for user `990f5837994f4a088c656c9b33e9a0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:58.487649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc7135d-91e8-4785-b922-d1639afed4cc service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:58.490350 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-61802c48-bc17-4ad1-903e-1b51d5db9bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:58.526892 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 875/1779] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:49:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:49:58.542440 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3314e-cb57-42ec-b963-54947416f04c None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:58.543310 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3314e-cb57-42ec-b963-54947416f04c None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:58.544292 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3314e-cb57-42ec-b963-54947416f04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:58.556991 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:58.612878 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-a4a3314e-cb57-42ec-b963-54947416f04c None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:58.620243 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a4a3314e-cb57-42ec-b963-54947416f04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:58.750607 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 905/1780] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Oct 3 12:49:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:58.753171 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61802c48-bc17-4ad1-903e-1b51d5db9bc3 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:58.756331 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-61802c48-bc17-4ad1-903e-1b51d5db9bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:58.802304 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 876/1781] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:58.913908 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3314e-cb57-42ec-b963-54947416f04c None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:58.914976 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a4a3314e-cb57-42ec-b963-54947416f04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:58.946841 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3314e-cb57-42ec-b963-54947416f04c None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:58.966726 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf67a2-72ae-4b76-a11f-298bea4e5851 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:58.967913 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf67a2-72ae-4b76-a11f-298bea4e5851 service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:58.969128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf67a2-72ae-4b76-a11f-298bea4e5851 service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:58.969407 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf67a2-72ae-4b76-a11f-298bea4e5851 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:58.981955 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61802c48-bc17-4ad1-903e-1b51d5db9bc3 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:58.990397 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbf67a2-72ae-4b76-a11f-298bea4e5851 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:58.993070 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbf67a2-72ae-4b76-a11f-298bea4e5851 service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:58.997027 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd8712-77cd-44b9-b7d6-6905458c5696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:58.997027 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd8712-77cd-44b9-b7d6-6905458c5696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:58.997027 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd8712-77cd-44b9-b7d6-6905458c5696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:58.997027 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd8712-77cd-44b9-b7d6-6905458c5696 admin admin] PATH_INFO: `/v3/projects/5cc4cb1efac849c3a9a00837f212f836` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:59.001476 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd8712-77cd-44b9-b7d6-6905458c5696 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cc4cb1efac849c3a9a00837f212f836)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:59.013350 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd8712-77cd-44b9-b7d6-6905458c5696 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:59.015974 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:59.021665 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 877/1782] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:49:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:49:59.022545 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf67a2-72ae-4b76-a11f-298bea4e5851 service nova] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:59.025545 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4bbf67a2-72ae-4b76-a11f-298bea4e5851 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:59.184280 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf67a2-72ae-4b76-a11f-298bea4e5851 service nova] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:59.212943 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-9cdd8712-77cd-44b9-b7d6-6905458c5696 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cc4cb1efac849c3a9a00837f212f836', 'request_id': 'req-9cdd8712-77cd-44b9-b7d6-6905458c5696'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:59.235636 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-9cdd8712-77cd-44b9-b7d6-6905458c5696 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cc4cb1efac849c3a9a00837f212f836', 'request_id': 'req-9cdd8712-77cd-44b9-b7d6-6905458c5696'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:59.322270 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a624552b-e2ca-4ce6-af17-f76b9b488b0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:59.322926 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-9cdd8712-77cd-44b9-b7d6-6905458c5696 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cc4cb1efac849c3a9a00837f212f836', 'request_id': 'req-9cdd8712-77cd-44b9-b7d6-6905458c5696'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:59.323408 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624552b-e2ca-4ce6-af17-f76b9b488b0f service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:59.323920 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624552b-e2ca-4ce6-af17-f76b9b488b0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:59.324335 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624552b-e2ca-4ce6-af17-f76b9b488b0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:59.339633 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-9cdd8712-77cd-44b9-b7d6-6905458c5696 admin admin] The token cache is being invalidate because project 5cc4cb1efac849c3a9a00837f212f836 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:49:59.344223 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-9cdd8712-77cd-44b9-b7d6-6905458c5696 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:59.348264 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-9cdd8712-77cd-44b9-b7d6-6905458c5696 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:59.355626 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 906/1783] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:49:58 2022] DELETE /identity/v3/projects/5cc4cb1efac849c3a9a00837f212f836 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:49:59.362895 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd8712-77cd-44b9-b7d6-6905458c5696 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:49:59.362895 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-9cdd8712-77cd-44b9-b7d6-6905458c5696 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:59.491698 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a624552b-e2ca-4ce6-af17-f76b9b488b0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:59.493798 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a624552b-e2ca-4ce6-af17-f76b9b488b0f service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:59.517788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:49:59.521398 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 878/1784] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:49:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:49:59.522953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26372d-9acb-465c-99b7-2bbfb663dc82 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:59.523248 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26372d-9acb-465c-99b7-2bbfb663dc82 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:59.523548 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26372d-9acb-465c-99b7-2bbfb663dc82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:59.586606 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-6f26372d-9acb-465c-99b7-2bbfb663dc82 None None] MFA Rules not processed for user `bd24d209dc9644e8b2860a0e676acd3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:49:59.593730 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-6f26372d-9acb-465c-99b7-2bbfb663dc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:49:59.607751 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd8712-77cd-44b9-b7d6-6905458c5696 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:49:59.647410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac346500-8925-4ced-b7f3-09f0d4d4f9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:49:59.651207 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac346500-8925-4ced-b7f3-09f0d4d4f9ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:49:59.651583 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac346500-8925-4ced-b7f3-09f0d4d4f9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:49:59.651867 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac346500-8925-4ced-b7f3-09f0d4d4f9ba admin admin] PATH_INFO: `/v3/users/26e8432d0cd5489d8af46fb774e93563` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:49:59.658594 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac346500-8925-4ced-b7f3-09f0d4d4f9ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=26e8432d0cd5489d8af46fb774e93563)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:49:59.665955 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac346500-8925-4ced-b7f3-09f0d4d4f9ba admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:49:59.781452 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 879/1785] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:49:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:49:59.887148 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-ac346500-8925-4ced-b7f3-09f0d4d4f9ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26e8432d0cd5489d8af46fb774e93563'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:59.896264 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-ac346500-8925-4ced-b7f3-09f0d4d4f9ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26e8432d0cd5489d8af46fb774e93563'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:59.919755 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-ac346500-8925-4ced-b7f3-09f0d4d4f9ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26e8432d0cd5489d8af46fb774e93563'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:59.920785 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-ac346500-8925-4ced-b7f3-09f0d4d4f9ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26e8432d0cd5489d8af46fb774e93563'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:49:59.940212 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:00.010028 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-ac346500-8925-4ced-b7f3-09f0d4d4f9ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26e8432d0cd5489d8af46fb774e93563'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:00.021199 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 907/1786] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:49:59 2022] DELETE /identity/v3/users/26e8432d0cd5489d8af46fb774e93563 => generated 0 bytes in 665 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:00.291129 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d351da-395a-4f02-8141-953dba1fafa8 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:00.292192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d351da-395a-4f02-8141-953dba1fafa8 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:00.292685 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d351da-395a-4f02-8141-953dba1fafa8 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:00.294799 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 880/1787] 10.222.0.33 () {54 vars in 921 bytes} [Mon Oct 3 12:50:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:50:00.350914 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b52f0-6b5b-48bf-9780-cecc39e0277a None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:00.351471 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b52f0-6b5b-48bf-9780-cecc39e0277a None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:00.351562 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b52f0-6b5b-48bf-9780-cecc39e0277a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:00.389771 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-732b52f0-6b5b-48bf-9780-cecc39e0277a None None] MFA Rules not processed for user `25eef64e92e9422ea62508c3eb0a197e`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:00.391026 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-732b52f0-6b5b-48bf-9780-cecc39e0277a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:00.526952 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 908/1788] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Oct 3 12:50:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:00.555388 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c2acdd-24f5-478b-8cee-5d08616985b5 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:00.555659 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c2acdd-24f5-478b-8cee-5d08616985b5 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:00.555941 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c2acdd-24f5-478b-8cee-5d08616985b5 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:00.557298 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 881/1789] 10.222.0.33 () {54 vars in 961 bytes} [Mon Oct 3 12:50:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:50:00.576481 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8481a-be97-434c-bdc5-26a8666a0b14 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:00.576979 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8481a-be97-434c-bdc5-26a8666a0b14 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:00.577476 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8481a-be97-434c-bdc5-26a8666a0b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:00.620521 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-65f8481a-be97-434c-bdc5-26a8666a0b14 None None] MFA Rules not processed for user `25eef64e92e9422ea62508c3eb0a197e`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:00.622793 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-65f8481a-be97-434c-bdc5-26a8666a0b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:00.655705 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c2acdd-24f5-478b-8cee-5d08616985b5 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:00.661055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 909/1790] 10.222.0.33 () {58 vars in 1058 bytes} [Mon Oct 3 12:50:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:00.692561 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c2acdd-24f5-478b-8cee-5d08616985b5 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:00.716213 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53945e01-f56a-4a99-a83f-5f0bc41424f8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25eef64e92e9422ea62508c3eb0a197e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:00.716213 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53945e01-f56a-4a99-a83f-5f0bc41424f8 service placement] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:00.716213 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53945e01-f56a-4a99-a83f-5f0bc41424f8 service placement] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:00.716213 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53945e01-f56a-4a99-a83f-5f0bc41424f8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:00.734873 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53945e01-f56a-4a99-a83f-5f0bc41424f8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:00.737809 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53945e01-f56a-4a99-a83f-5f0bc41424f8 service placement] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:00.767002 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 882/1791] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:50:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:50:00.908018 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:00.924516 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f7be4a-34d7-4fd8-99ed-80be959ab66c None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:00.924516 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f7be4a-34d7-4fd8-99ed-80be959ab66c None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:00.924516 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f7be4a-34d7-4fd8-99ed-80be959ab66c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:00.959682 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-46f7be4a-34d7-4fd8-99ed-80be959ab66c None None] MFA Rules not processed for user `676b2a6bbf6e4df1b47dba2a9b8e8862`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:00.961081 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-46f7be4a-34d7-4fd8-99ed-80be959ab66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:01.007673 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:01.081300 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 910/1792] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:00 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:01.283376 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:02.001053 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53945e01-f56a-4a99-a83f-5f0bc41424f8 service placement] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:02.003678 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-53945e01-f56a-4a99-a83f-5f0bc41424f8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:02.141304 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53945e01-f56a-4a99-a83f-5f0bc41424f8 service placement] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:02.150788 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce36c46b-ebd3-41c7-a31f-78ba9ff6da44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:02.151879 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36c46b-ebd3-41c7-a31f-78ba9ff6da44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:02.152153 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36c46b-ebd3-41c7-a31f-78ba9ff6da44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:02.152496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36c46b-ebd3-41c7-a31f-78ba9ff6da44 admin admin] PATH_INFO: `/v3/projects/1c505a5b81c8490c899486f892830a8a` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:02.155266 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce36c46b-ebd3-41c7-a31f-78ba9ff6da44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c505a5b81c8490c899486f892830a8a)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:02.158368 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce36c46b-ebd3-41c7-a31f-78ba9ff6da44 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:02.348177 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ce36c46b-ebd3-41c7-a31f-78ba9ff6da44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c505a5b81c8490c899486f892830a8a', 'request_id': 'req-ce36c46b-ebd3-41c7-a31f-78ba9ff6da44'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:02.370681 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ce36c46b-ebd3-41c7-a31f-78ba9ff6da44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c505a5b81c8490c899486f892830a8a', 'request_id': 'req-ce36c46b-ebd3-41c7-a31f-78ba9ff6da44'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:02.478323 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ce36c46b-ebd3-41c7-a31f-78ba9ff6da44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c505a5b81c8490c899486f892830a8a', 'request_id': 'req-ce36c46b-ebd3-41c7-a31f-78ba9ff6da44'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:02.510178 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ce36c46b-ebd3-41c7-a31f-78ba9ff6da44 admin admin] The token cache is being invalidate because project 1c505a5b81c8490c899486f892830a8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:50:02.510468 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ce36c46b-ebd3-41c7-a31f-78ba9ff6da44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:02.511953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ce36c46b-ebd3-41c7-a31f-78ba9ff6da44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:02.517556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 883/1793] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:50:01 2022] DELETE /identity/v3/projects/1c505a5b81c8490c899486f892830a8a => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:02.522573 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f7be4a-34d7-4fd8-99ed-80be959ab66c None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:02.551384 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f7be4a-34d7-4fd8-99ed-80be959ab66c None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:02.562520 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce36c46b-ebd3-41c7-a31f-78ba9ff6da44 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:02.563651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ce36c46b-ebd3-41c7-a31f-78ba9ff6da44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:02.691758 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:02.733143 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f532d1-e785-4b7a-acce-a6c06c8eb330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:02.733369 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce36c46b-ebd3-41c7-a31f-78ba9ff6da44 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:02.734341 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f532d1-e785-4b7a-acce-a6c06c8eb330 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:02.734804 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f532d1-e785-4b7a-acce-a6c06c8eb330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:02.735537 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f532d1-e785-4b7a-acce-a6c06c8eb330 admin admin] PATH_INFO: `/v3/users/9c44405819d94af3968bdd99921df3c7` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:02.738910 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f532d1-e785-4b7a-acce-a6c06c8eb330 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c44405819d94af3968bdd99921df3c7)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:02.741023 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f532d1-e785-4b7a-acce-a6c06c8eb330 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:02.754361 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a73cdd-eaa6-42ff-9384-3a5a38e4391f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:02.776781 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a73cdd-eaa6-42ff-9384-3a5a38e4391f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:02.776781 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a73cdd-eaa6-42ff-9384-3a5a38e4391f admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:02.776781 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a73cdd-eaa6-42ff-9384-3a5a38e4391f admin admin] PATH_INFO: `/v3/users/8741999af245405dabc5f8a02032ca11` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:02.776781 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a73cdd-eaa6-42ff-9384-3a5a38e4391f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8741999af245405dabc5f8a02032ca11)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:02.776781 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a73cdd-eaa6-42ff-9384-3a5a38e4391f admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:03.070289 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d5a73cdd-eaa6-42ff-9384-3a5a38e4391f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8741999af245405dabc5f8a02032ca11'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:03.104724 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d5a73cdd-eaa6-42ff-9384-3a5a38e4391f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8741999af245405dabc5f8a02032ca11'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:03.109409 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d5a73cdd-eaa6-42ff-9384-3a5a38e4391f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8741999af245405dabc5f8a02032ca11'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:03.124198 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-c3f532d1-e785-4b7a-acce-a6c06c8eb330 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c44405819d94af3968bdd99921df3c7'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:03.130280 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d5a73cdd-eaa6-42ff-9384-3a5a38e4391f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8741999af245405dabc5f8a02032ca11'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:03.146736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-c3f532d1-e785-4b7a-acce-a6c06c8eb330 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c44405819d94af3968bdd99921df3c7'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:03.177669 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-c3f532d1-e785-4b7a-acce-a6c06c8eb330 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c44405819d94af3968bdd99921df3c7'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:03.179331 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-c3f532d1-e785-4b7a-acce-a6c06c8eb330 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c44405819d94af3968bdd99921df3c7'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:03.222141 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d5a73cdd-eaa6-42ff-9384-3a5a38e4391f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8741999af245405dabc5f8a02032ca11'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:03.270143 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 884/1794] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:50:02 2022] DELETE /identity/v3/users/8741999af245405dabc5f8a02032ca11 => generated 0 bytes in 702 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:50:03.302822 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a73cdd-eaa6-42ff-9384-3a5a38e4391f admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:03.307850 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-c3f532d1-e785-4b7a-acce-a6c06c8eb330 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c44405819d94af3968bdd99921df3c7'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:03.330173 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 911/1795] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:50:02 2022] DELETE /identity/v3/users/9c44405819d94af3968bdd99921df3c7 => generated 0 bytes in 812 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:03.331471 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d5a73cdd-eaa6-42ff-9384-3a5a38e4391f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:03.503031 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a73cdd-eaa6-42ff-9384-3a5a38e4391f admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:03.672205 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341991d7-4acc-40db-be1f-e8e9218e3f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:03.673074 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341991d7-4acc-40db-be1f-e8e9218e3f26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:03.674366 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341991d7-4acc-40db-be1f-e8e9218e3f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:03.675393 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341991d7-4acc-40db-be1f-e8e9218e3f26 admin admin] PATH_INFO: `/v3/projects/57aad6f1fc244cdcba3b5a62b904eb50` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:03.677169 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341991d7-4acc-40db-be1f-e8e9218e3f26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57aad6f1fc244cdcba3b5a62b904eb50)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:03.680401 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341991d7-4acc-40db-be1f-e8e9218e3f26 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:03.709664 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:03.795753 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f532d1-e785-4b7a-acce-a6c06c8eb330 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:03.802884 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-c3f532d1-e785-4b7a-acce-a6c06c8eb330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:03.842558 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f532d1-e785-4b7a-acce-a6c06c8eb330 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:03.862260 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:03.889199 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-341991d7-4acc-40db-be1f-e8e9218e3f26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57aad6f1fc244cdcba3b5a62b904eb50', 'request_id': 'req-341991d7-4acc-40db-be1f-e8e9218e3f26'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:03.909831 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-341991d7-4acc-40db-be1f-e8e9218e3f26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57aad6f1fc244cdcba3b5a62b904eb50', 'request_id': 'req-341991d7-4acc-40db-be1f-e8e9218e3f26'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:03.976341 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed5c002-91d2-4745-ad7e-eb5e67c427d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:03.979183 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5c002-91d2-4745-ad7e-eb5e67c427d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:03.979680 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5c002-91d2-4745-ad7e-eb5e67c427d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:03.980106 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5c002-91d2-4745-ad7e-eb5e67c427d0 admin admin] PATH_INFO: `/v3/projects/c88366da05654b2c8edbb4261b8a90be` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:03.982193 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed5c002-91d2-4745-ad7e-eb5e67c427d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c88366da05654b2c8edbb4261b8a90be)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:03.988806 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed5c002-91d2-4745-ad7e-eb5e67c427d0 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:04.010247 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-341991d7-4acc-40db-be1f-e8e9218e3f26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57aad6f1fc244cdcba3b5a62b904eb50', 'request_id': 'req-341991d7-4acc-40db-be1f-e8e9218e3f26'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:04.042690 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-341991d7-4acc-40db-be1f-e8e9218e3f26 admin admin] The token cache is being invalidate because project 57aad6f1fc244cdcba3b5a62b904eb50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:50:04.044921 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-341991d7-4acc-40db-be1f-e8e9218e3f26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:04.051864 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-341991d7-4acc-40db-be1f-e8e9218e3f26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:04.060924 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 885/1796] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:50:03 2022] DELETE /identity/v3/projects/57aad6f1fc244cdcba3b5a62b904eb50 => generated 0 bytes in 790 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:04.068113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49c946-e465-4370-b8d9-593e5a9dfc7b None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:04.068397 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49c946-e465-4370-b8d9-593e5a9dfc7b None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:04.068677 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49c946-e465-4370-b8d9-593e5a9dfc7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:04.109372 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-6e49c946-e465-4370-b8d9-593e5a9dfc7b None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:04.111529 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-6e49c946-e465-4370-b8d9-593e5a9dfc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:04.122024 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 886/1797] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:50:04.131341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a01d1-b7c2-44ac-bb8f-1eb1f06a162f None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:04.134271 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a01d1-b7c2-44ac-bb8f-1eb1f06a162f None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:04.135438 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a01d1-b7c2-44ac-bb8f-1eb1f06a162f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:04.189661 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-302a01d1-b7c2-44ac-bb8f-1eb1f06a162f None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:04.189661 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-302a01d1-b7c2-44ac-bb8f-1eb1f06a162f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:04.227847 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-6ed5c002-91d2-4745-ad7e-eb5e67c427d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c88366da05654b2c8edbb4261b8a90be', 'request_id': 'req-6ed5c002-91d2-4745-ad7e-eb5e67c427d0'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:04.260188 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-6ed5c002-91d2-4745-ad7e-eb5e67c427d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c88366da05654b2c8edbb4261b8a90be', 'request_id': 'req-6ed5c002-91d2-4745-ad7e-eb5e67c427d0'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:04.340777 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 887/1798] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:04.341858 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a01d1-b7c2-44ac-bb8f-1eb1f06a162f None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:04.352281 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a01d1-b7c2-44ac-bb8f-1eb1f06a162f None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:04.358599 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282ae9d-d9a3-4cbb-a474-3813a578428a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:04.359636 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282ae9d-d9a3-4cbb-a474-3813a578428a admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:04.360024 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282ae9d-d9a3-4cbb-a474-3813a578428a admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:04.360398 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282ae9d-d9a3-4cbb-a474-3813a578428a admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:04.361804 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a282ae9d-d9a3-4cbb-a474-3813a578428a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:50:04.362436 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a282ae9d-d9a3-4cbb-a474-3813a578428a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:04.364162 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a282ae9d-d9a3-4cbb-a474-3813a578428a admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:04.394687 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 888/1799] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:50:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:50:04.417738 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282ae9d-d9a3-4cbb-a474-3813a578428a admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:04.422440 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282ae9d-d9a3-4cbb-a474-3813a578428a admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:04.436044 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046e3d49-3aa2-488a-bb73-ef0b53b94dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:04.436665 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e3d49-3aa2-488a-bb73-ef0b53b94dd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:04.436941 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e3d49-3aa2-488a-bb73-ef0b53b94dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:04.437213 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e3d49-3aa2-488a-bb73-ef0b53b94dd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:04.437848 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046e3d49-3aa2-488a-bb73-ef0b53b94dd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:04.439838 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046e3d49-3aa2-488a-bb73-ef0b53b94dd6 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:04.534291 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-6ed5c002-91d2-4745-ad7e-eb5e67c427d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c88366da05654b2c8edbb4261b8a90be', 'request_id': 'req-6ed5c002-91d2-4745-ad7e-eb5e67c427d0'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:04.545670 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-6ed5c002-91d2-4745-ad7e-eb5e67c427d0 admin admin] The token cache is being invalidate because project c88366da05654b2c8edbb4261b8a90be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:50:04.546260 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-6ed5c002-91d2-4745-ad7e-eb5e67c427d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:04.547248 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-6ed5c002-91d2-4745-ad7e-eb5e67c427d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:04.549112 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 912/1800] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:50:03 2022] DELETE /identity/v3/projects/c88366da05654b2c8edbb4261b8a90be => generated 0 bytes in 754 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:50:04.552998 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed5c002-91d2-4745-ad7e-eb5e67c427d0 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:04.554380 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-6ed5c002-91d2-4745-ad7e-eb5e67c427d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:04.567010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed5c002-91d2-4745-ad7e-eb5e67c427d0 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:04.571622 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3eda4-b043-4163-9c96-34e702984c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:04.575171 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3eda4-b043-4163-9c96-34e702984c3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:04.575171 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3eda4-b043-4163-9c96-34e702984c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:04.575171 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3eda4-b043-4163-9c96-34e702984c3e admin admin] PATH_INFO: `/v3/users/3f1a5651effd4245938c04420647adf5` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:04.575171 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d3eda4-b043-4163-9c96-34e702984c3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f1a5651effd4245938c04420647adf5)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:04.575958 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d3eda4-b043-4163-9c96-34e702984c3e admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:04.605512 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 889/1801] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:50:04 2022] POST /identity/v3/projects => generated 377 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:50:04.606624 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046e3d49-3aa2-488a-bb73-ef0b53b94dd6 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:04.609946 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-046e3d49-3aa2-488a-bb73-ef0b53b94dd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:04.713549 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046e3d49-3aa2-488a-bb73-ef0b53b94dd6 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:04.720360 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1242982-9aea-43f8-be2a-96a8f506ec6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:04.721390 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1242982-9aea-43f8-be2a-96a8f506ec6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:04.721761 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1242982-9aea-43f8-be2a-96a8f506ec6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:04.722042 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1242982-9aea-43f8-be2a-96a8f506ec6c admin admin] PATH_INFO: `/v3/projects/4ed0f7ec4435426eafce871fe179dc7a` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:04.724190 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1242982-9aea-43f8-be2a-96a8f506ec6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ed0f7ec4435426eafce871fe179dc7a)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:04.730412 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1242982-9aea-43f8-be2a-96a8f506ec6c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:04.732217 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:04.825552 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-74d3eda4-b043-4163-9c96-34e702984c3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f1a5651effd4245938c04420647adf5'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:04.834827 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-74d3eda4-b043-4163-9c96-34e702984c3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f1a5651effd4245938c04420647adf5'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:04.848940 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-74d3eda4-b043-4163-9c96-34e702984c3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f1a5651effd4245938c04420647adf5'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:04.849706 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-74d3eda4-b043-4163-9c96-34e702984c3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f1a5651effd4245938c04420647adf5'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:04.895439 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e1242982-9aea-43f8-be2a-96a8f506ec6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ed0f7ec4435426eafce871fe179dc7a', 'request_id': 'req-e1242982-9aea-43f8-be2a-96a8f506ec6c'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:04.901071 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e1242982-9aea-43f8-be2a-96a8f506ec6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ed0f7ec4435426eafce871fe179dc7a', 'request_id': 'req-e1242982-9aea-43f8-be2a-96a8f506ec6c'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:04.956843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-74d3eda4-b043-4163-9c96-34e702984c3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f1a5651effd4245938c04420647adf5'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:04.967174 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 913/1802] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:50:04 2022] DELETE /identity/v3/users/3f1a5651effd4245938c04420647adf5 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:50:04.968571 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3eda4-b043-4163-9c96-34e702984c3e admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:04.969836 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-74d3eda4-b043-4163-9c96-34e702984c3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:04.991596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:05.041940 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e1242982-9aea-43f8-be2a-96a8f506ec6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ed0f7ec4435426eafce871fe179dc7a', 'request_id': 'req-e1242982-9aea-43f8-be2a-96a8f506ec6c'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:05.063585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e1242982-9aea-43f8-be2a-96a8f506ec6c admin admin] The token cache is being invalidate because project 4ed0f7ec4435426eafce871fe179dc7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:50:05.064155 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e1242982-9aea-43f8-be2a-96a8f506ec6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:05.065892 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e1242982-9aea-43f8-be2a-96a8f506ec6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:05.070133 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3eda4-b043-4163-9c96-34e702984c3e admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:05.070384 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 890/1803] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:50:04 2022] DELETE /identity/v3/projects/4ed0f7ec4435426eafce871fe179dc7a => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:05.083396 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d057f5ce-86f9-4a0c-967a-59a93af45ecf None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:05.084034 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d057f5ce-86f9-4a0c-967a-59a93af45ecf None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:05.084500 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d057f5ce-86f9-4a0c-967a-59a93af45ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:05.093395 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8048e-82a5-4d80-a372-a9e661358fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:05.094034 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8048e-82a5-4d80-a372-a9e661358fb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:05.094315 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8048e-82a5-4d80-a372-a9e661358fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:05.094601 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8048e-82a5-4d80-a372-a9e661358fb5 admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:05.096506 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8048e-82a5-4d80-a372-a9e661358fb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:05.097288 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8048e-82a5-4d80-a372-a9e661358fb5 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:05.123182 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-d057f5ce-86f9-4a0c-967a-59a93af45ecf None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:05.124510 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d057f5ce-86f9-4a0c-967a-59a93af45ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:05.135703 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 891/1804] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:50:05.143541 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37a03e-1687-4513-bf30-97919391134d None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:05.143848 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37a03e-1687-4513-bf30-97919391134d None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:05.144618 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37a03e-1687-4513-bf30-97919391134d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:05.176810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-da37a03e-1687-4513-bf30-97919391134d None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:05.179271 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-da37a03e-1687-4513-bf30-97919391134d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:05.190194 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 892/1805] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:50:05.194450 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da37a03e-1687-4513-bf30-97919391134d None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:05.203942 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da37a03e-1687-4513-bf30-97919391134d None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:05.209534 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07751523-d41d-4966-af0a-b50e2b7fad12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:05.210269 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07751523-d41d-4966-af0a-b50e2b7fad12 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:05.210646 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07751523-d41d-4966-af0a-b50e2b7fad12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:05.210983 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07751523-d41d-4966-af0a-b50e2b7fad12 admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:05.211729 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07751523-d41d-4966-af0a-b50e2b7fad12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:50:05.212104 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07751523-d41d-4966-af0a-b50e2b7fad12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:05.213694 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07751523-d41d-4966-af0a-b50e2b7fad12 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:05.225876 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 893/1806] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:50:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:50:05.229594 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 914/1807] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:50:04 2022] POST /identity/v3/users => generated 289 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:05.230407 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07751523-d41d-4966-af0a-b50e2b7fad12 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:05.230657 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8048e-82a5-4d80-a372-a9e661358fb5 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:05.232331 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-d5c8048e-82a5-4d80-a372-a9e661358fb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:05.241118 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07751523-d41d-4966-af0a-b50e2b7fad12 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:05.246184 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaaed5e-cd02-46d2-8431-c5615a83e25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:05.246332 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8048e-82a5-4d80-a372-a9e661358fb5 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:05.247230 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaaed5e-cd02-46d2-8431-c5615a83e25e admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:05.247230 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaaed5e-cd02-46d2-8431-c5615a83e25e admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:05.255371 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaaed5e-cd02-46d2-8431-c5615a83e25e admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:05.255371 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaaed5e-cd02-46d2-8431-c5615a83e25e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:05.255371 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaaed5e-cd02-46d2-8431-c5615a83e25e admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:05.261171 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a75825-ac3d-4772-ba0b-c056bd112f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:05.262013 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a75825-ac3d-4772-ba0b-c056bd112f78 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:05.262584 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a75825-ac3d-4772-ba0b-c056bd112f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:05.263706 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a75825-ac3d-4772-ba0b-c056bd112f78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:05.264580 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a75825-ac3d-4772-ba0b-c056bd112f78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:05.266529 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a75825-ac3d-4772-ba0b-c056bd112f78 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:05.281683 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 915/1808] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:05 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:50:05.282549 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a75825-ac3d-4772-ba0b-c056bd112f78 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:05.285735 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-27a75825-ac3d-4772-ba0b-c056bd112f78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:05.299218 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a75825-ac3d-4772-ba0b-c056bd112f78 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:05.304901 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e396c257-9d92-4f7a-818a-a3d405585384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:05.306910 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e396c257-9d92-4f7a-818a-a3d405585384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:05.306910 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e396c257-9d92-4f7a-818a-a3d405585384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:05.306910 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e396c257-9d92-4f7a-818a-a3d405585384 admin admin] PATH_INFO: `/v3/projects/1eb3e2e4a6c1421f9f5ce7fae68051ad` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:05.307292 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e396c257-9d92-4f7a-818a-a3d405585384 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1eb3e2e4a6c1421f9f5ce7fae68051ad)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:05.309373 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e396c257-9d92-4f7a-818a-a3d405585384 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:05.382062 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 894/1809] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:50:05 2022] POST /identity/v3/projects => generated 393 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:05.383131 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaaed5e-cd02-46d2-8431-c5615a83e25e admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:05.424663 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-e396c257-9d92-4f7a-818a-a3d405585384 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1eb3e2e4a6c1421f9f5ce7fae68051ad', 'request_id': 'req-e396c257-9d92-4f7a-818a-a3d405585384'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:05.433129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-e396c257-9d92-4f7a-818a-a3d405585384 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1eb3e2e4a6c1421f9f5ce7fae68051ad', 'request_id': 'req-e396c257-9d92-4f7a-818a-a3d405585384'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:05.453073 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaaed5e-cd02-46d2-8431-c5615a83e25e admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:05.503118 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:05.516423 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123034db-0c64-4e52-9f47-ce70e7d47ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:05.522175 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123034db-0c64-4e52-9f47-ce70e7d47ef1 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:05.522175 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123034db-0c64-4e52-9f47-ce70e7d47ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:05.522175 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123034db-0c64-4e52-9f47-ce70e7d47ef1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:05.522175 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123034db-0c64-4e52-9f47-ce70e7d47ef1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:05.522175 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123034db-0c64-4e52-9f47-ce70e7d47ef1 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:05.535951 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 895/1810] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:05 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:50:05.536898 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123034db-0c64-4e52-9f47-ce70e7d47ef1 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:05.540890 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-e396c257-9d92-4f7a-818a-a3d405585384 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1eb3e2e4a6c1421f9f5ce7fae68051ad', 'request_id': 'req-e396c257-9d92-4f7a-818a-a3d405585384'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:05.548377 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123034db-0c64-4e52-9f47-ce70e7d47ef1 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:05.559803 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-e396c257-9d92-4f7a-818a-a3d405585384 admin admin] The token cache is being invalidate because project 1eb3e2e4a6c1421f9f5ce7fae68051ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:50:05.560925 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-e396c257-9d92-4f7a-818a-a3d405585384 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:05.562294 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-e396c257-9d92-4f7a-818a-a3d405585384 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:05.563375 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39d1f5-a67f-4ede-874d-45734e651938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:05.567530 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39d1f5-a67f-4ede-874d-45734e651938 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:05.567530 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39d1f5-a67f-4ede-874d-45734e651938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:05.568225 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39d1f5-a67f-4ede-874d-45734e651938 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:05.569484 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 916/1811] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:50:05 2022] DELETE /identity/v3/projects/1eb3e2e4a6c1421f9f5ce7fae68051ad => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:05.570504 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e396c257-9d92-4f7a-818a-a3d405585384 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:05.571087 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39d1f5-a67f-4ede-874d-45734e651938 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:05.573830 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-e396c257-9d92-4f7a-818a-a3d405585384 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:05.576085 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39d1f5-a67f-4ede-874d-45734e651938 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:05.611590 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e396c257-9d92-4f7a-818a-a3d405585384 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:05.616736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a24f36-e805-48ff-8891-f053557506b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:05.617472 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a24f36-e805-48ff-8891-f053557506b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:05.617814 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a24f36-e805-48ff-8891-f053557506b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:05.618149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a24f36-e805-48ff-8891-f053557506b3 admin admin] PATH_INFO: `/v3/projects/2364d42ae14344caa1abfbab7816d28b/users/40b84af5a0cb410f8bb6fe48fb56e66a/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:05.634848 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a24f36-e805-48ff-8891-f053557506b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2364d42ae14344caa1abfbab7816d28b, user_id=40b84af5a0cb410f8bb6fe48fb56e66a, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:05.637571 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a24f36-e805-48ff-8891-f053557506b3 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:05.711936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 896/1812] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:50:05 2022] POST /identity/v3/users => generated 297 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:05.713197 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a911c-1978-4abb-abf3-9c612a64410d None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:05.713492 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a911c-1978-4abb-abf3-9c612a64410d None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:05.713770 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a911c-1978-4abb-abf3-9c612a64410d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:05.733310 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-cc6a911c-1978-4abb-abf3-9c612a64410d None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:05.734501 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-cc6a911c-1978-4abb-abf3-9c612a64410d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:05.741581 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 917/1813] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:50:05 2022] PUT /identity/v3/projects/2364d42ae14344caa1abfbab7816d28b/users/40b84af5a0cb410f8bb6fe48fb56e66a/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:05.745501 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a24f36-e805-48ff-8891-f053557506b3 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:05.745501 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-58a24f36-e805-48ff-8891-f053557506b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:05.803885 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:05.835202 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 897/1814] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:05.837275 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2245853-1de8-4e8c-ab64-866d315e74e6 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:05.837565 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2245853-1de8-4e8c-ab64-866d315e74e6 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:05.837840 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2245853-1de8-4e8c-ab64-866d315e74e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:05.840606 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a24f36-e805-48ff-8891-f053557506b3 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:05.846917 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6374be2d-32b0-43b0-9d90-7516d4366ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:05.847664 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374be2d-32b0-43b0-9d90-7516d4366ec7 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:05.847951 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374be2d-32b0-43b0-9d90-7516d4366ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:05.848229 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374be2d-32b0-43b0-9d90-7516d4366ec7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:05.848918 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6374be2d-32b0-43b0-9d90-7516d4366ec7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:05.850536 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6374be2d-32b0-43b0-9d90-7516d4366ec7 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:05.862769 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 918/1815] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:05 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:50:05.864244 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc522042-01b5-4776-a3df-19ee6ca7e71d None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:05.864536 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc522042-01b5-4776-a3df-19ee6ca7e71d None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:05.864816 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc522042-01b5-4776-a3df-19ee6ca7e71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:05.871273 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-d2245853-1de8-4e8c-ab64-866d315e74e6 None None] MFA Rules not processed for user `40b84af5a0cb410f8bb6fe48fb56e66a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:05.872486 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d2245853-1de8-4e8c-ab64-866d315e74e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:05.891234 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-cc522042-01b5-4776-a3df-19ee6ca7e71d None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:05.894230 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-cc522042-01b5-4776-a3df-19ee6ca7e71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:05.901125 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 919/1816] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:05.902060 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc522042-01b5-4776-a3df-19ee6ca7e71d None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:05.912061 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc522042-01b5-4776-a3df-19ee6ca7e71d None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:05.917203 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61576fb8-1375-4aac-9310-cef9b625d149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:05.917843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61576fb8-1375-4aac-9310-cef9b625d149 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:05.919576 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61576fb8-1375-4aac-9310-cef9b625d149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:05.919576 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61576fb8-1375-4aac-9310-cef9b625d149 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:05.919576 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61576fb8-1375-4aac-9310-cef9b625d149 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:05.920652 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61576fb8-1375-4aac-9310-cef9b625d149 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:05.934413 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 920/1817] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:50:05.934413 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61576fb8-1375-4aac-9310-cef9b625d149 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:05.942639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61576fb8-1375-4aac-9310-cef9b625d149 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:05.946374 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 898/1818] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:05.947180 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044911ec-95e7-4d43-bb45-a662dfb698f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:05.947300 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2245853-1de8-4e8c-ab64-866d315e74e6 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:05.951704 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044911ec-95e7-4d43-bb45-a662dfb698f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:05.951704 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044911ec-95e7-4d43-bb45-a662dfb698f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:05.951704 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044911ec-95e7-4d43-bb45-a662dfb698f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:05.951704 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044911ec-95e7-4d43-bb45-a662dfb698f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:50:05.951704 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044911ec-95e7-4d43-bb45-a662dfb698f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:05.951704 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044911ec-95e7-4d43-bb45-a662dfb698f0 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:05.957998 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2245853-1de8-4e8c-ab64-866d315e74e6 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:05.962713 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2839932a-cf9d-4633-9471-a51c41474303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:05.963399 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2839932a-cf9d-4633-9471-a51c41474303 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:05.963686 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2839932a-cf9d-4633-9471-a51c41474303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:05.963962 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2839932a-cf9d-4633-9471-a51c41474303 admin admin] PATH_INFO: `/v3/projects/f04332e6d0ce41ca823ea6c2c7e47ed7/users/97b2a559b7b64db09e8db8b163ba3704/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:05.964958 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 921/1819] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:50:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:50:05.965744 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044911ec-95e7-4d43-bb45-a662dfb698f0 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:05.975523 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044911ec-95e7-4d43-bb45-a662dfb698f0 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:05.979072 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2839932a-cf9d-4633-9471-a51c41474303 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f04332e6d0ce41ca823ea6c2c7e47ed7, user_id=97b2a559b7b64db09e8db8b163ba3704, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:05.980556 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dddaca6-d452-43dd-a755-ed254062850b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:05.981206 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dddaca6-d452-43dd-a755-ed254062850b admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:05.986878 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dddaca6-d452-43dd-a755-ed254062850b admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:05.986878 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dddaca6-d452-43dd-a755-ed254062850b admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:05.987706 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2839932a-cf9d-4633-9471-a51c41474303 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:05.987790 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dddaca6-d452-43dd-a755-ed254062850b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:05.987790 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dddaca6-d452-43dd-a755-ed254062850b admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:06.082906 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 899/1820] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:50:05 2022] PUT /identity/v3/projects/f04332e6d0ce41ca823ea6c2c7e47ed7/users/97b2a559b7b64db09e8db8b163ba3704/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:06.083675 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2839932a-cf9d-4633-9471-a51c41474303 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:06.109312 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 922/1821] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:50:05 2022] POST /identity/v3/projects => generated 375 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:06.117297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5865c4-745d-47b1-8264-11351341cece None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:06.117297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5865c4-745d-47b1-8264-11351341cece None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:06.117297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5865c4-745d-47b1-8264-11351341cece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:06.129621 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:06.154780 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2839932a-cf9d-4633-9471-a51c41474303 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:06.163276 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-1f5865c4-745d-47b1-8264-11351341cece None None] MFA Rules not processed for user `97b2a559b7b64db09e8db8b163ba3704`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:06.164550 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-1f5865c4-745d-47b1-8264-11351341cece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:06.226672 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086c3c9-fdae-42d3-b9dc-7abe9a56a9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:06.227309 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086c3c9-fdae-42d3-b9dc-7abe9a56a9dd admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:06.228144 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086c3c9-fdae-42d3-b9dc-7abe9a56a9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:06.228422 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086c3c9-fdae-42d3-b9dc-7abe9a56a9dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:06.229108 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d086c3c9-fdae-42d3-b9dc-7abe9a56a9dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:06.231055 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d086c3c9-fdae-42d3-b9dc-7abe9a56a9dd admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:06.247905 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 923/1822] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:06 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:06.248797 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5865c4-745d-47b1-8264-11351341cece None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:06.266074 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5865c4-745d-47b1-8264-11351341cece None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:06.272470 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800391f4-d5e0-4e33-a31f-7ae9095bc045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:06.273108 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800391f4-d5e0-4e33-a31f-7ae9095bc045 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:06.273358 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800391f4-d5e0-4e33-a31f-7ae9095bc045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:06.273627 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800391f4-d5e0-4e33-a31f-7ae9095bc045 admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:06.274399 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800391f4-d5e0-4e33-a31f-7ae9095bc045 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:06.276539 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800391f4-d5e0-4e33-a31f-7ae9095bc045 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:06.324339 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 900/1823] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:50:06 2022] POST /identity/v3/projects => generated 381 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:06.325224 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086c3c9-fdae-42d3-b9dc-7abe9a56a9dd admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:06.353543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 924/1824] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:50:06 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:06.353543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800391f4-d5e0-4e33-a31f-7ae9095bc045 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:06.357433 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800391f4-d5e0-4e33-a31f-7ae9095bc045 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:06.400277 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086c3c9-fdae-42d3-b9dc-7abe9a56a9dd admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:06.405567 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b85ff0-acce-4de2-ac3d-013b28aeca05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:06.406633 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b85ff0-acce-4de2-ac3d-013b28aeca05 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:06.406917 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b85ff0-acce-4de2-ac3d-013b28aeca05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:06.407528 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b85ff0-acce-4de2-ac3d-013b28aeca05 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:06.408278 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b85ff0-acce-4de2-ac3d-013b28aeca05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:06.410507 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b85ff0-acce-4de2-ac3d-013b28aeca05 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:06.427492 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0878e46d-4e56-477a-8ce6-df1fce86d373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:06.432231 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878e46d-4e56-477a-8ce6-df1fce86d373 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:06.432231 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878e46d-4e56-477a-8ce6-df1fce86d373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:06.432231 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878e46d-4e56-477a-8ce6-df1fce86d373 admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:06.432231 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0878e46d-4e56-477a-8ce6-df1fce86d373 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:06.432231 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0878e46d-4e56-477a-8ce6-df1fce86d373 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:06.441781 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:06.518377 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 901/1825] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:50:06 2022] POST /identity/v3/projects => generated 393 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:06.519246 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b85ff0-acce-4de2-ac3d-013b28aeca05 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:06.530101 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 925/1826] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:50:06 2022] POST /identity/v3/users => generated 291 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:06.535527 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0878e46d-4e56-477a-8ce6-df1fce86d373 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:06.541348 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0878e46d-4e56-477a-8ce6-df1fce86d373 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:06.572580 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:06.583145 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b85ff0-acce-4de2-ac3d-013b28aeca05 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:06.588007 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eb0e6a-77ba-47d9-b3a3-00a170a47753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:06.588643 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb0e6a-77ba-47d9-b3a3-00a170a47753 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:06.589123 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb0e6a-77ba-47d9-b3a3-00a170a47753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:06.589401 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb0e6a-77ba-47d9-b3a3-00a170a47753 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:06.590047 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eb0e6a-77ba-47d9-b3a3-00a170a47753 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:06.591678 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eb0e6a-77ba-47d9-b3a3-00a170a47753 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:06.601114 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c4e21c-39f4-4785-9a4b-afcd88ae7210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:06.601736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4e21c-39f4-4785-9a4b-afcd88ae7210 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:06.605484 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 902/1827] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:50:06.605603 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4e21c-39f4-4785-9a4b-afcd88ae7210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:06.605603 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4e21c-39f4-4785-9a4b-afcd88ae7210 admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:06.605748 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eb0e6a-77ba-47d9-b3a3-00a170a47753 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:06.605824 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c4e21c-39f4-4785-9a4b-afcd88ae7210 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:06.605824 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c4e21c-39f4-4785-9a4b-afcd88ae7210 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:06.615716 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eb0e6a-77ba-47d9-b3a3-00a170a47753 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:06.621509 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d3d78-1de0-45a7-abbe-dd1fb47f4cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:06.622124 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d3d78-1de0-45a7-abbe-dd1fb47f4cdf admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:06.622408 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d3d78-1de0-45a7-abbe-dd1fb47f4cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:06.622690 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d3d78-1de0-45a7-abbe-dd1fb47f4cdf admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:06.623356 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514d3d78-1de0-45a7-abbe-dd1fb47f4cdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:06.624991 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514d3d78-1de0-45a7-abbe-dd1fb47f4cdf admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:06.638792 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 903/1828] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:50:06.638792 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d3d78-1de0-45a7-abbe-dd1fb47f4cdf admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:06.648978 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d3d78-1de0-45a7-abbe-dd1fb47f4cdf admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:06.655127 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c690fe9e-46ba-4f5f-bd0e-360fdadefa7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:06.655127 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690fe9e-46ba-4f5f-bd0e-360fdadefa7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:06.655127 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690fe9e-46ba-4f5f-bd0e-360fdadefa7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:06.655447 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690fe9e-46ba-4f5f-bd0e-360fdadefa7c admin admin] PATH_INFO: `/v3/projects/5c9d5ae85ee042a28915126e0e04b9d4/users/27d21574ca16490581dfe9edce146952/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:06.674855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c690fe9e-46ba-4f5f-bd0e-360fdadefa7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c9d5ae85ee042a28915126e0e04b9d4, user_id=27d21574ca16490581dfe9edce146952, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:06.678400 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c690fe9e-46ba-4f5f-bd0e-360fdadefa7c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:06.700219 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 926/1829] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:50:06 2022] POST /identity/v3/users => generated 303 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:06.701506 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c4e21c-39f4-4785-9a4b-afcd88ae7210 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:06.713429 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c4e21c-39f4-4785-9a4b-afcd88ae7210 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:06.718148 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a985d9e-5a1d-4df8-9be8-a9a388ac6b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:06.718751 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a985d9e-5a1d-4df8-9be8-a9a388ac6b65 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:06.719032 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a985d9e-5a1d-4df8-9be8-a9a388ac6b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:06.719388 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a985d9e-5a1d-4df8-9be8-a9a388ac6b65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:06.724296 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a985d9e-5a1d-4df8-9be8-a9a388ac6b65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:06.724296 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a985d9e-5a1d-4df8-9be8-a9a388ac6b65 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:06.738078 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 927/1830] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:50:06.738078 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a985d9e-5a1d-4df8-9be8-a9a388ac6b65 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:06.750450 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a985d9e-5a1d-4df8-9be8-a9a388ac6b65 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:06.756390 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a4ddc-f839-4ed3-b895-7e29ec958424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:06.757078 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a4ddc-f839-4ed3-b895-7e29ec958424 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:06.757424 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a4ddc-f839-4ed3-b895-7e29ec958424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:06.757762 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a4ddc-f839-4ed3-b895-7e29ec958424 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:06.758468 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325a4ddc-f839-4ed3-b895-7e29ec958424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:06.760203 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325a4ddc-f839-4ed3-b895-7e29ec958424 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:06.770817 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 904/1831] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:50:06 2022] PUT /identity/v3/projects/5c9d5ae85ee042a28915126e0e04b9d4/users/27d21574ca16490581dfe9edce146952/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:06.771845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 928/1832] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:50:06.772677 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c690fe9e-46ba-4f5f-bd0e-360fdadefa7c admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:06.776148 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a4ddc-f839-4ed3-b895-7e29ec958424 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:06.787664 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a4ddc-f839-4ed3-b895-7e29ec958424 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:06.851564 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c690fe9e-46ba-4f5f-bd0e-360fdadefa7c admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:06.852441 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364af974-73a2-459f-aa6c-7643437223bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:06.853033 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364af974-73a2-459f-aa6c-7643437223bd admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:06.853588 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364af974-73a2-459f-aa6c-7643437223bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:06.853858 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364af974-73a2-459f-aa6c-7643437223bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:06.854801 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364af974-73a2-459f-aa6c-7643437223bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:06.856444 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78972357-3676-4f02-a9a2-9da6c3098d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:06.856573 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364af974-73a2-459f-aa6c-7643437223bd admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:06.857057 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78972357-3676-4f02-a9a2-9da6c3098d17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:06.857332 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78972357-3676-4f02-a9a2-9da6c3098d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:06.857608 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78972357-3676-4f02-a9a2-9da6c3098d17 admin admin] PATH_INFO: `/v3/projects/24908375f07b4dbfbb23cfdf8b8692dc/users/9c3e9709384f44d4981ddbed49476564/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:06.868504 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 929/1833] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:50:06.869470 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364af974-73a2-459f-aa6c-7643437223bd admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:06.875570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78972357-3676-4f02-a9a2-9da6c3098d17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24908375f07b4dbfbb23cfdf8b8692dc, user_id=9c3e9709384f44d4981ddbed49476564, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:06.880475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78972357-3676-4f02-a9a2-9da6c3098d17 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:06.882310 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364af974-73a2-459f-aa6c-7643437223bd admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:06.887548 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc9449-f3c1-43a2-baa4-addae853242a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:06.888199 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc9449-f3c1-43a2-baa4-addae853242a admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:06.888476 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc9449-f3c1-43a2-baa4-addae853242a admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:06.888745 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc9449-f3c1-43a2-baa4-addae853242a admin admin] PATH_INFO: `/v3/projects/a2da15bfe6274a6cab836f1786a847fe/users/5a0c6feb988a4408946ee2de8096837f/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:06.906982 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afc9449-f3c1-43a2-baa4-addae853242a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2da15bfe6274a6cab836f1786a847fe, user_id=5a0c6feb988a4408946ee2de8096837f, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:06.909893 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afc9449-f3c1-43a2-baa4-addae853242a admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:06.963047 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 905/1834] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:50:06 2022] PUT /identity/v3/projects/24908375f07b4dbfbb23cfdf8b8692dc/users/9c3e9709384f44d4981ddbed49476564/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:06.964868 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78972357-3676-4f02-a9a2-9da6c3098d17 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:06.992783 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 930/1835] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:50:06 2022] PUT /identity/v3/projects/a2da15bfe6274a6cab836f1786a847fe/users/5a0c6feb988a4408946ee2de8096837f/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:06.996506 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7bf72-3345-48a5-afe8-68379ad3edba None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:06.996506 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7bf72-3345-48a5-afe8-68379ad3edba None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:06.996506 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7bf72-3345-48a5-afe8-68379ad3edba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:07.023003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-2bb7bf72-3345-48a5-afe8-68379ad3edba None None] MFA Rules not processed for user `9c3e9709384f44d4981ddbed49476564`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:07.024312 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-2bb7bf72-3345-48a5-afe8-68379ad3edba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:07.040234 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78972357-3676-4f02-a9a2-9da6c3098d17 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:07.044704 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:07.083586 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:07.090829 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 931/1836] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:07.091727 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb7bf72-3345-48a5-afe8-68379ad3edba None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:07.099564 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8dfaaf-ef8a-475b-8b78-9b588a77ab87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:07.100270 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8dfaaf-ef8a-475b-8b78-9b588a77ab87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:07.100555 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8dfaaf-ef8a-475b-8b78-9b588a77ab87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:07.100831 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8dfaaf-ef8a-475b-8b78-9b588a77ab87 admin admin] PATH_INFO: `/v3/projects/5c9d5ae85ee042a28915126e0e04b9d4/users/27d21574ca16490581dfe9edce146952/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:07.102781 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8dfaaf-ef8a-475b-8b78-9b588a77ab87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c9d5ae85ee042a28915126e0e04b9d4, user_id=27d21574ca16490581dfe9edce146952, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:07.105497 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8dfaaf-ef8a-475b-8b78-9b588a77ab87 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:07.116426 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 906/1837] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:50:06 2022] PUT /identity/v3/projects/5c9d5ae85ee042a28915126e0e04b9d4/users/27d21574ca16490581dfe9edce146952/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:07.118947 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8dfaaf-ef8a-475b-8b78-9b588a77ab87 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:07.154775 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb7bf72-3345-48a5-afe8-68379ad3edba None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:07.188675 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8dfaaf-ef8a-475b-8b78-9b588a77ab87 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:07.193962 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712cd0a-7db2-4da2-9024-c29e4c87c1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:07.195008 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712cd0a-7db2-4da2-9024-c29e4c87c1fd admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:07.195656 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712cd0a-7db2-4da2-9024-c29e4c87c1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:07.195935 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712cd0a-7db2-4da2-9024-c29e4c87c1fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:07.196573 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4712cd0a-7db2-4da2-9024-c29e4c87c1fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:07.198448 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4712cd0a-7db2-4da2-9024-c29e4c87c1fd admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:07.211430 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef8cbdd-398e-4251-b961-354e35437628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:07.212054 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8cbdd-398e-4251-b961-354e35437628 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:07.212339 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8cbdd-398e-4251-b961-354e35437628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:07.212616 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8cbdd-398e-4251-b961-354e35437628 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:07.213259 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef8cbdd-398e-4251-b961-354e35437628 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:07.214856 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef8cbdd-398e-4251-b961-354e35437628 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:07.224791 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 932/1838] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:50:07.225978 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892066c6-300c-4821-a04f-82764587ec7b None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:07.226265 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892066c6-300c-4821-a04f-82764587ec7b None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:07.226555 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892066c6-300c-4821-a04f-82764587ec7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:07.256549 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-892066c6-300c-4821-a04f-82764587ec7b None None] MFA Rules not processed for user `27d21574ca16490581dfe9edce146952`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:07.257775 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-892066c6-300c-4821-a04f-82764587ec7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:07.272749 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 907/1839] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:50:07 2022] POST /identity/v3/projects => generated 379 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:07.273570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712cd0a-7db2-4da2-9024-c29e4c87c1fd admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:07.310979 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:07.330733 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 933/1840] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:07 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:07.332343 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892066c6-300c-4821-a04f-82764587ec7b None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:07.345953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712cd0a-7db2-4da2-9024-c29e4c87c1fd admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:07.350908 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892066c6-300c-4821-a04f-82764587ec7b None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:07.351723 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc905a5-5981-4908-a51a-f971e4b144a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:07.352490 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc905a5-5981-4908-a51a-f971e4b144a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:07.352831 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc905a5-5981-4908-a51a-f971e4b144a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:07.353116 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc905a5-5981-4908-a51a-f971e4b144a8 admin admin] PATH_INFO: `/v3/projects/a2da15bfe6274a6cab836f1786a847fe/users/5a0c6feb988a4408946ee2de8096837f/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:07.356735 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc905a5-5981-4908-a51a-f971e4b144a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2da15bfe6274a6cab836f1786a847fe, user_id=5a0c6feb988a4408946ee2de8096837f, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:07.356856 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b907f191-a2e1-425c-b207-c97502f560e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:07.358385 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907f191-a2e1-425c-b207-c97502f560e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:07.358675 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907f191-a2e1-425c-b207-c97502f560e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:07.358951 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907f191-a2e1-425c-b207-c97502f560e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:07.359689 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b907f191-a2e1-425c-b207-c97502f560e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:07.362522 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b907f191-a2e1-425c-b207-c97502f560e7 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:07.367661 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc905a5-5981-4908-a51a-f971e4b144a8 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:07.384248 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 908/1841] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:50:07 2022] PUT /identity/v3/projects/a2da15bfe6274a6cab836f1786a847fe/users/5a0c6feb988a4408946ee2de8096837f/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:07.384868 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc905a5-5981-4908-a51a-f971e4b144a8 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:07.386652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1bc905a5-5981-4908-a51a-f971e4b144a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:07.452499 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 934/1842] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:50:07 2022] POST /identity/v3/users => generated 296 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:07.452642 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04abd7-9df5-482d-87a6-233674b01a01 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:07.452949 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04abd7-9df5-482d-87a6-233674b01a01 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:07.453512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04abd7-9df5-482d-87a6-233674b01a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:07.497541 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc905a5-5981-4908-a51a-f971e4b144a8 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:07.504570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd36179-9d49-4880-9949-3832eb766829 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:07.504570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd36179-9d49-4880-9949-3832eb766829 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:07.504570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd36179-9d49-4880-9949-3832eb766829 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:07.504570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd36179-9d49-4880-9949-3832eb766829 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:07.514947 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:07.527149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-6a04abd7-9df5-482d-87a6-233674b01a01 None None] MFA Rules not processed for user `5a0c6feb988a4408946ee2de8096837f`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:07.535477 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-6a04abd7-9df5-482d-87a6-233674b01a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:07.593611 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd36179-9d49-4880-9949-3832eb766829 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:07.595044 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd36179-9d49-4880-9949-3832eb766829 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:07.611822 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 909/1843] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:50:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:50:07.611822 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd36179-9d49-4880-9949-3832eb766829 service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:07.646787 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 935/1844] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:07 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:07.681316 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a04abd7-9df5-482d-87a6-233674b01a01 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:07.682746 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-6a04abd7-9df5-482d-87a6-233674b01a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:07.695999 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a04abd7-9df5-482d-87a6-233674b01a01 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:07.700463 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d73e6-cd10-45f8-b20f-d6f9fa20503a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:07.701080 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d73e6-cd10-45f8-b20f-d6f9fa20503a service cinder] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:07.701362 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d73e6-cd10-45f8-b20f-d6f9fa20503a service cinder] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:07.701634 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d73e6-cd10-45f8-b20f-d6f9fa20503a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:07.711300 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd36179-9d49-4880-9949-3832eb766829 service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:07.712143 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24d73e6-cd10-45f8-b20f-d6f9fa20503a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:07.713578 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24d73e6-cd10-45f8-b20f-d6f9fa20503a service cinder] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:07.716350 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1fbc63-2cc6-457a-a3db-6be5520bafc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:07.717380 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1fbc63-2cc6-457a-a3db-6be5520bafc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:07.717853 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1fbc63-2cc6-457a-a3db-6be5520bafc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:07.717967 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1fbc63-2cc6-457a-a3db-6be5520bafc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:07.719486 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1fbc63-2cc6-457a-a3db-6be5520bafc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:07.725073 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1fbc63-2cc6-457a-a3db-6be5520bafc0 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:07.727749 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 936/1845] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:50:07 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:50:07.733215 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 910/1846] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:50:07.737626 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d73e6-cd10-45f8-b20f-d6f9fa20503a service cinder] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:07.751279 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d73e6-cd10-45f8-b20f-d6f9fa20503a service cinder] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:07.757912 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ce8744-fdfd-470a-b5a5-08094065c108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:07.758784 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce8744-fdfd-470a-b5a5-08094065c108 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:07.759186 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce8744-fdfd-470a-b5a5-08094065c108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:07.759562 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce8744-fdfd-470a-b5a5-08094065c108 admin admin] PATH_INFO: `/v3/projects/7183d55deb42413c89dfd47336a8cae3/users/2e3f8ead49c94b0ba4a6502b594ad556/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:07.782075 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ce8744-fdfd-470a-b5a5-08094065c108 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7183d55deb42413c89dfd47336a8cae3, user_id=2e3f8ead49c94b0ba4a6502b594ad556, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:07.785610 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ce8744-fdfd-470a-b5a5-08094065c108 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:07.849870 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 937/1847] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:50:07 2022] PUT /identity/v3/projects/7183d55deb42413c89dfd47336a8cae3/users/2e3f8ead49c94b0ba4a6502b594ad556/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:07.854448 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1fbc63-2cc6-457a-a3db-6be5520bafc0 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:07.867532 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1fbc63-2cc6-457a-a3db-6be5520bafc0 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:07.884980 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ce8744-fdfd-470a-b5a5-08094065c108 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:07.956199 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ce8744-fdfd-470a-b5a5-08094065c108 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:07.960482 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa1c871-ff75-44aa-93a8-909f78079fb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:07.961102 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa1c871-ff75-44aa-93a8-909f78079fb5 service cinder] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:07.961409 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa1c871-ff75-44aa-93a8-909f78079fb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:07.961713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa1c871-ff75-44aa-93a8-909f78079fb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:07.965988 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b1a1a9-b61b-4162-8cf2-48122e22e817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:07.967024 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b1a1a9-b61b-4162-8cf2-48122e22e817 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:07.967545 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b1a1a9-b61b-4162-8cf2-48122e22e817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:07.967821 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b1a1a9-b61b-4162-8cf2-48122e22e817 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:07.968470 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b1a1a9-b61b-4162-8cf2-48122e22e817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:07.970103 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b1a1a9-b61b-4162-8cf2-48122e22e817 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:07.983520 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 911/1848] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:50:07.988634 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b1a1a9-b61b-4162-8cf2-48122e22e817 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:07.999766 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b1a1a9-b61b-4162-8cf2-48122e22e817 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:08.005513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:08.007617 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f23dd78-6bb5-4c8a-a0ff-3c9e6f4565b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:08.008287 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23dd78-6bb5-4c8a-a0ff-3c9e6f4565b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:08.008611 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23dd78-6bb5-4c8a-a0ff-3c9e6f4565b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:08.008843 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23dd78-6bb5-4c8a-a0ff-3c9e6f4565b5 admin admin] PATH_INFO: `/v3/projects/7183d55deb42413c89dfd47336a8cae3/users/2e3f8ead49c94b0ba4a6502b594ad556/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:08.011030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f23dd78-6bb5-4c8a-a0ff-3c9e6f4565b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7183d55deb42413c89dfd47336a8cae3, user_id=2e3f8ead49c94b0ba4a6502b594ad556, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:08.013783 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f23dd78-6bb5-4c8a-a0ff-3c9e6f4565b5 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:08.033154 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 912/1849] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:50:07 2022] PUT /identity/v3/projects/7183d55deb42413c89dfd47336a8cae3/users/2e3f8ead49c94b0ba4a6502b594ad556/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:50:08.038771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094b838-8a81-4e11-8d14-5c3366717fa4 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:08.044563 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094b838-8a81-4e11-8d14-5c3366717fa4 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:08.044563 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094b838-8a81-4e11-8d14-5c3366717fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:08.056545 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa1c871-ff75-44aa-93a8-909f78079fb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:08.059584 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa1c871-ff75-44aa-93a8-909f78079fb5 service cinder] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:08.083467 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 938/1850] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:50:07 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:50:08.090248 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-b094b838-8a81-4e11-8d14-5c3366717fa4 None None] MFA Rules not processed for user `2e3f8ead49c94b0ba4a6502b594ad556`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:08.091535 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b094b838-8a81-4e11-8d14-5c3366717fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:08.113084 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:08.193060 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 913/1851] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:08 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:50:08.502968 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa1c871-ff75-44aa-93a8-909f78079fb5 service cinder] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:08.513033 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa1c871-ff75-44aa-93a8-909f78079fb5 service cinder] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:08.566678 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fba45c-5048-4adc-82ef-2e167ffdfc0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:08.567693 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fba45c-5048-4adc-82ef-2e167ffdfc0c service cinder] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:08.568070 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fba45c-5048-4adc-82ef-2e167ffdfc0c service cinder] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:08.568350 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fba45c-5048-4adc-82ef-2e167ffdfc0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:08.578427 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fba45c-5048-4adc-82ef-2e167ffdfc0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:08.580679 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fba45c-5048-4adc-82ef-2e167ffdfc0c service cinder] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:08.590954 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 939/1852] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:50:08 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:50:09.746409 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b094b838-8a81-4e11-8d14-5c3366717fa4 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:09.756554 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b094b838-8a81-4e11-8d14-5c3366717fa4 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:09.761684 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a9958-580e-4e7e-90a7-8f75c5daef7d tempest-VolumeQuotaClassesTest-278968539 tempest-VolumeQuotaClassesTest-278968539-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3f8ead49c94b0ba4a6502b594ad556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7183d55deb42413c89dfd47336a8cae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:09.762303 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a9958-580e-4e7e-90a7-8f75c5daef7d tempest-VolumeQuotaClassesTest-278968539 tempest-VolumeQuotaClassesTest-278968539-project-admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:09.762610 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a9958-580e-4e7e-90a7-8f75c5daef7d tempest-VolumeQuotaClassesTest-278968539 tempest-VolumeQuotaClassesTest-278968539-project-admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:09.762885 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a9958-580e-4e7e-90a7-8f75c5daef7d tempest-VolumeQuotaClassesTest-278968539 tempest-VolumeQuotaClassesTest-278968539-project-admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:09.763558 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35a9958-580e-4e7e-90a7-8f75c5daef7d tempest-VolumeQuotaClassesTest-278968539 tempest-VolumeQuotaClassesTest-278968539-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:50:09.764219 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35a9958-580e-4e7e-90a7-8f75c5daef7d tempest-VolumeQuotaClassesTest-278968539 tempest-VolumeQuotaClassesTest-278968539-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:09.765451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35a9958-580e-4e7e-90a7-8f75c5daef7d tempest-VolumeQuotaClassesTest-278968539 tempest-VolumeQuotaClassesTest-278968539-project-admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:09.778955 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 914/1853] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:50:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:50:09.785251 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fba45c-5048-4adc-82ef-2e167ffdfc0c service cinder] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:09.796118 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fba45c-5048-4adc-82ef-2e167ffdfc0c service cinder] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:09.803613 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b7d56-deb6-4f2b-aee9-1c17694a56b0 tempest-VolumeQuotaClassesTest-278968539 tempest-VolumeQuotaClassesTest-278968539-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3f8ead49c94b0ba4a6502b594ad556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7183d55deb42413c89dfd47336a8cae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:09.804445 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b7d56-deb6-4f2b-aee9-1c17694a56b0 tempest-VolumeQuotaClassesTest-278968539 tempest-VolumeQuotaClassesTest-278968539-project-admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:09.926260 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b7d56-deb6-4f2b-aee9-1c17694a56b0 tempest-VolumeQuotaClassesTest-278968539 tempest-VolumeQuotaClassesTest-278968539-project-admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:09.926260 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b7d56-deb6-4f2b-aee9-1c17694a56b0 tempest-VolumeQuotaClassesTest-278968539 tempest-VolumeQuotaClassesTest-278968539-project-admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:09.926260 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38b7d56-deb6-4f2b-aee9-1c17694a56b0 tempest-VolumeQuotaClassesTest-278968539 tempest-VolumeQuotaClassesTest-278968539-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:09.926260 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38b7d56-deb6-4f2b-aee9-1c17694a56b0 tempest-VolumeQuotaClassesTest-278968539 tempest-VolumeQuotaClassesTest-278968539-project-admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:10.812624 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 940/1854] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:50:09 2022] POST /identity/v3/projects => generated 354 bytes in 1028 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:50:10.817254 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a9958-580e-4e7e-90a7-8f75c5daef7d tempest-VolumeQuotaClassesTest-278968539 tempest-VolumeQuotaClassesTest-278968539-project-admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:10.827660 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a9958-580e-4e7e-90a7-8f75c5daef7d tempest-VolumeQuotaClassesTest-278968539 tempest-VolumeQuotaClassesTest-278968539-project-admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:10.983375 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a7d23-5754-4daf-aaed-44d093e61792 tempest-VolumeQuotaClassesTest-278968539 tempest-VolumeQuotaClassesTest-278968539-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3f8ead49c94b0ba4a6502b594ad556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7183d55deb42413c89dfd47336a8cae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:10.983375 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a7d23-5754-4daf-aaed-44d093e61792 tempest-VolumeQuotaClassesTest-278968539 tempest-VolumeQuotaClassesTest-278968539-project-admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:10.983375 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a7d23-5754-4daf-aaed-44d093e61792 tempest-VolumeQuotaClassesTest-278968539 tempest-VolumeQuotaClassesTest-278968539-project-admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:10.984343 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a7d23-5754-4daf-aaed-44d093e61792 tempest-VolumeQuotaClassesTest-278968539 tempest-VolumeQuotaClassesTest-278968539-project-admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:10.984343 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a7d23-5754-4daf-aaed-44d093e61792 tempest-VolumeQuotaClassesTest-278968539 tempest-VolumeQuotaClassesTest-278968539-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:50:10.984343 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a7d23-5754-4daf-aaed-44d093e61792 tempest-VolumeQuotaClassesTest-278968539 tempest-VolumeQuotaClassesTest-278968539-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:10.984343 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a7d23-5754-4daf-aaed-44d093e61792 tempest-VolumeQuotaClassesTest-278968539 tempest-VolumeQuotaClassesTest-278968539-project-admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:11.008967 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 915/1855] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:50:11.369421 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b7d56-deb6-4f2b-aee9-1c17694a56b0 tempest-VolumeQuotaClassesTest-278968539 tempest-VolumeQuotaClassesTest-278968539-project-admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:11.395669 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b7d56-deb6-4f2b-aee9-1c17694a56b0 tempest-VolumeQuotaClassesTest-278968539 tempest-VolumeQuotaClassesTest-278968539-project-admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:11.409517 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fcf88d-430c-45f9-8dd8-7e7a712e2926 tempest-VolumeQuotaClassesTest-278968539 tempest-VolumeQuotaClassesTest-278968539-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3f8ead49c94b0ba4a6502b594ad556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7183d55deb42413c89dfd47336a8cae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:11.410226 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcf88d-430c-45f9-8dd8-7e7a712e2926 tempest-VolumeQuotaClassesTest-278968539 tempest-VolumeQuotaClassesTest-278968539-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:11.410520 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcf88d-430c-45f9-8dd8-7e7a712e2926 tempest-VolumeQuotaClassesTest-278968539 tempest-VolumeQuotaClassesTest-278968539-project-admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:11.411359 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcf88d-430c-45f9-8dd8-7e7a712e2926 tempest-VolumeQuotaClassesTest-278968539 tempest-VolumeQuotaClassesTest-278968539-project-admin] PATH_INFO: `/v3/projects/5d5a1a24c3604adaa351d1029d00faf0` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:11.412800 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fcf88d-430c-45f9-8dd8-7e7a712e2926 tempest-VolumeQuotaClassesTest-278968539 tempest-VolumeQuotaClassesTest-278968539-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5d5a1a24c3604adaa351d1029d00faf0)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:11.415456 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fcf88d-430c-45f9-8dd8-7e7a712e2926 tempest-VolumeQuotaClassesTest-278968539 tempest-VolumeQuotaClassesTest-278968539-project-admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:11.582249 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-75fcf88d-430c-45f9-8dd8-7e7a712e2926 tempest-VolumeQuotaClassesTest-278968539 tempest-VolumeQuotaClassesTest-278968539-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d5a1a24c3604adaa351d1029d00faf0', 'request_id': 'req-75fcf88d-430c-45f9-8dd8-7e7a712e2926'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:11.591230 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-75fcf88d-430c-45f9-8dd8-7e7a712e2926 tempest-VolumeQuotaClassesTest-278968539 tempest-VolumeQuotaClassesTest-278968539-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d5a1a24c3604adaa351d1029d00faf0', 'request_id': 'req-75fcf88d-430c-45f9-8dd8-7e7a712e2926'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:11.700104 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-75fcf88d-430c-45f9-8dd8-7e7a712e2926 tempest-VolumeQuotaClassesTest-278968539 tempest-VolumeQuotaClassesTest-278968539-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d5a1a24c3604adaa351d1029d00faf0', 'request_id': 'req-75fcf88d-430c-45f9-8dd8-7e7a712e2926'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:11.728292 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-75fcf88d-430c-45f9-8dd8-7e7a712e2926 tempest-VolumeQuotaClassesTest-278968539 tempest-VolumeQuotaClassesTest-278968539-project-admin] The token cache is being invalidate because project 5d5a1a24c3604adaa351d1029d00faf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:50:11.728800 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-75fcf88d-430c-45f9-8dd8-7e7a712e2926 tempest-VolumeQuotaClassesTest-278968539 tempest-VolumeQuotaClassesTest-278968539-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:11.729627 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-75fcf88d-430c-45f9-8dd8-7e7a712e2926 tempest-VolumeQuotaClassesTest-278968539 tempest-VolumeQuotaClassesTest-278968539-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:11.731267 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 941/1856] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:50:11 2022] DELETE /identity/v3/projects/5d5a1a24c3604adaa351d1029d00faf0 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:12.783352 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a7d23-5754-4daf-aaed-44d093e61792 tempest-VolumeQuotaClassesTest-278968539 tempest-VolumeQuotaClassesTest-278968539-project-admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:12.785324 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ca7a7d23-5754-4daf-aaed-44d093e61792 tempest-VolumeQuotaClassesTest-278968539 tempest-VolumeQuotaClassesTest-278968539-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:12.851038 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a7d23-5754-4daf-aaed-44d093e61792 tempest-VolumeQuotaClassesTest-278968539 tempest-VolumeQuotaClassesTest-278968539-project-admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:12.904754 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:12.928792 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4defc61-60fa-4e78-bf94-1d0e514998cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:12.929678 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4defc61-60fa-4e78-bf94-1d0e514998cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:12.929960 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4defc61-60fa-4e78-bf94-1d0e514998cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:12.930236 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4defc61-60fa-4e78-bf94-1d0e514998cc admin admin] PATH_INFO: `/v3/users/9c3e9709384f44d4981ddbed49476564` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:12.931408 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4defc61-60fa-4e78-bf94-1d0e514998cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c3e9709384f44d4981ddbed49476564)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:12.935094 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4defc61-60fa-4e78-bf94-1d0e514998cc admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:13.071253 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d4defc61-60fa-4e78-bf94-1d0e514998cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c3e9709384f44d4981ddbed49476564'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:13.083737 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d4defc61-60fa-4e78-bf94-1d0e514998cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c3e9709384f44d4981ddbed49476564'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:13.084457 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d4defc61-60fa-4e78-bf94-1d0e514998cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c3e9709384f44d4981ddbed49476564'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:13.091526 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d4defc61-60fa-4e78-bf94-1d0e514998cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c3e9709384f44d4981ddbed49476564'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:13.156830 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d4defc61-60fa-4e78-bf94-1d0e514998cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c3e9709384f44d4981ddbed49476564'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:13.163541 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 916/1857] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:50:12 2022] DELETE /identity/v3/users/9c3e9709384f44d4981ddbed49476564 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:13.168064 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f48288-7e57-4062-8d90-ebe5cf3657ce None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:13.168564 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f48288-7e57-4062-8d90-ebe5cf3657ce None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:13.168651 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f48288-7e57-4062-8d90-ebe5cf3657ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:13.187463 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-81f48288-7e57-4062-8d90-ebe5cf3657ce None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:13.188659 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-81f48288-7e57-4062-8d90-ebe5cf3657ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:13.224338 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:13.272725 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 942/1858] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:13.279087 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4defc61-60fa-4e78-bf94-1d0e514998cc admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:13.280528 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d4defc61-60fa-4e78-bf94-1d0e514998cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:13.348365 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4defc61-60fa-4e78-bf94-1d0e514998cc admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:13.350372 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c3b8d-d971-420d-96a3-0019028764f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:13.351100 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c3b8d-d971-420d-96a3-0019028764f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:13.351495 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c3b8d-d971-420d-96a3-0019028764f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:13.351833 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c3b8d-d971-420d-96a3-0019028764f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:13.361681 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014c3b8d-d971-420d-96a3-0019028764f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:13.363405 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014c3b8d-d971-420d-96a3-0019028764f8 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:13.373462 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 917/1859] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:50:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:50:13.788407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f48288-7e57-4062-8d90-ebe5cf3657ce None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:13.789822 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-81f48288-7e57-4062-8d90-ebe5cf3657ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:13.804059 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f48288-7e57-4062-8d90-ebe5cf3657ce None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:13.810663 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df00abd-1c6c-434b-bfd8-f1190c4adb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:13.811487 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df00abd-1c6c-434b-bfd8-f1190c4adb65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:13.811695 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df00abd-1c6c-434b-bfd8-f1190c4adb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:13.812068 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df00abd-1c6c-434b-bfd8-f1190c4adb65 admin admin] PATH_INFO: `/v3/projects/24908375f07b4dbfbb23cfdf8b8692dc` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:13.813269 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df00abd-1c6c-434b-bfd8-f1190c4adb65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24908375f07b4dbfbb23cfdf8b8692dc)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:13.816051 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df00abd-1c6c-434b-bfd8-f1190c4adb65 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:13.936989 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-2df00abd-1c6c-434b-bfd8-f1190c4adb65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24908375f07b4dbfbb23cfdf8b8692dc', 'request_id': 'req-2df00abd-1c6c-434b-bfd8-f1190c4adb65'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:13.944942 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-2df00abd-1c6c-434b-bfd8-f1190c4adb65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24908375f07b4dbfbb23cfdf8b8692dc', 'request_id': 'req-2df00abd-1c6c-434b-bfd8-f1190c4adb65'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:14.065746 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-2df00abd-1c6c-434b-bfd8-f1190c4adb65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24908375f07b4dbfbb23cfdf8b8692dc', 'request_id': 'req-2df00abd-1c6c-434b-bfd8-f1190c4adb65'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:14.076732 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-2df00abd-1c6c-434b-bfd8-f1190c4adb65 admin admin] The token cache is being invalidate because project 24908375f07b4dbfbb23cfdf8b8692dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:50:14.077018 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-2df00abd-1c6c-434b-bfd8-f1190c4adb65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:14.077669 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-2df00abd-1c6c-434b-bfd8-f1190c4adb65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:14.079161 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 943/1860] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:50:13 2022] DELETE /identity/v3/projects/24908375f07b4dbfbb23cfdf8b8692dc => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:14.083464 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c3b8d-d971-420d-96a3-0019028764f8 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:14.093647 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c3b8d-d971-420d-96a3-0019028764f8 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:14.162335 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828ec0b-9e26-4d4c-8335-e8950b184b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:14.163042 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828ec0b-9e26-4d4c-8335-e8950b184b53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:14.163401 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828ec0b-9e26-4d4c-8335-e8950b184b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:14.171244 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828ec0b-9e26-4d4c-8335-e8950b184b53 admin admin] PATH_INFO: `/v3/users/2e3f8ead49c94b0ba4a6502b594ad556` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:14.171244 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7828ec0b-9e26-4d4c-8335-e8950b184b53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e3f8ead49c94b0ba4a6502b594ad556)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:14.171244 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7828ec0b-9e26-4d4c-8335-e8950b184b53 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:14.173923 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:14.388184 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7828ec0b-9e26-4d4c-8335-e8950b184b53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e3f8ead49c94b0ba4a6502b594ad556'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:14.401724 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7828ec0b-9e26-4d4c-8335-e8950b184b53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e3f8ead49c94b0ba4a6502b594ad556'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:14.402160 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7828ec0b-9e26-4d4c-8335-e8950b184b53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e3f8ead49c94b0ba4a6502b594ad556'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:14.409996 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7828ec0b-9e26-4d4c-8335-e8950b184b53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e3f8ead49c94b0ba4a6502b594ad556'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:14.522742 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7828ec0b-9e26-4d4c-8335-e8950b184b53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e3f8ead49c94b0ba4a6502b594ad556'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:14.529431 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 918/1861] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:50:14 2022] DELETE /identity/v3/users/2e3f8ead49c94b0ba4a6502b594ad556 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:14.891211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df00abd-1c6c-434b-bfd8-f1190c4adb65 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:14.893530 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-2df00abd-1c6c-434b-bfd8-f1190c4adb65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:14.968094 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df00abd-1c6c-434b-bfd8-f1190c4adb65 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:15.028926 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ddf51-9633-42bd-b7bc-62bc00ef08df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:15.029630 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ddf51-9633-42bd-b7bc-62bc00ef08df service cinder] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:15.030340 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ddf51-9633-42bd-b7bc-62bc00ef08df service cinder] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:15.030340 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ddf51-9633-42bd-b7bc-62bc00ef08df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:15.032065 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-e06ddf51-9633-42bd-b7bc-62bc00ef08df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:15.059090 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828ec0b-9e26-4d4c-8335-e8950b184b53 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:15.060568 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-7828ec0b-9e26-4d4c-8335-e8950b184b53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:15.099181 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:15.137290 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06ddf51-9633-42bd-b7bc-62bc00ef08df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:15.139070 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06ddf51-9633-42bd-b7bc-62bc00ef08df service cinder] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:15.152745 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 944/1862] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:50:14 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:50:15.160014 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828ec0b-9e26-4d4c-8335-e8950b184b53 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:15.165320 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06271b-388b-4da1-ae90-4ff120b61fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:15.165988 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06271b-388b-4da1-ae90-4ff120b61fd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:15.166922 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06271b-388b-4da1-ae90-4ff120b61fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:15.166922 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06271b-388b-4da1-ae90-4ff120b61fd0 admin admin] PATH_INFO: `/v3/projects/7183d55deb42413c89dfd47336a8cae3` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:15.168127 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd06271b-388b-4da1-ae90-4ff120b61fd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7183d55deb42413c89dfd47336a8cae3)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:15.170622 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd06271b-388b-4da1-ae90-4ff120b61fd0 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:15.427706 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-dd06271b-388b-4da1-ae90-4ff120b61fd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7183d55deb42413c89dfd47336a8cae3', 'request_id': 'req-dd06271b-388b-4da1-ae90-4ff120b61fd0'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:15.427706 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-dd06271b-388b-4da1-ae90-4ff120b61fd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7183d55deb42413c89dfd47336a8cae3', 'request_id': 'req-dd06271b-388b-4da1-ae90-4ff120b61fd0'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:15.505652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-dd06271b-388b-4da1-ae90-4ff120b61fd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7183d55deb42413c89dfd47336a8cae3', 'request_id': 'req-dd06271b-388b-4da1-ae90-4ff120b61fd0'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:15.520622 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-dd06271b-388b-4da1-ae90-4ff120b61fd0 admin admin] The token cache is being invalidate because project 7183d55deb42413c89dfd47336a8cae3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:50:15.521169 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-dd06271b-388b-4da1-ae90-4ff120b61fd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:15.521718 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-dd06271b-388b-4da1-ae90-4ff120b61fd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:15.523725 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 919/1863] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:50:15 2022] DELETE /identity/v3/projects/7183d55deb42413c89dfd47336a8cae3 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:50:15.528446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5566c-4066-48a5-b1d5-c3875d8175b0 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:15.528938 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5566c-4066-48a5-b1d5-c3875d8175b0 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:15.529358 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5566c-4066-48a5-b1d5-c3875d8175b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:15.551051 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-09a5566c-4066-48a5-b1d5-c3875d8175b0 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:15.552768 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-09a5566c-4066-48a5-b1d5-c3875d8175b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:15.631373 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 945/1864] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:15.639723 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc75a5-ae8b-41aa-bace-d198fa9896a4 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:15.639723 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc75a5-ae8b-41aa-bace-d198fa9896a4 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:15.642544 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc75a5-ae8b-41aa-bace-d198fa9896a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:15.675207 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127e410-cefc-4435-a10a-3b46c41b644b None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:15.675848 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127e410-cefc-4435-a10a-3b46c41b644b None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:15.676269 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127e410-cefc-4435-a10a-3b46c41b644b None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:15.677775 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 946/1865] 10.222.0.33 () {54 vars in 921 bytes} [Mon Oct 3 12:50:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 12:50:15.690993 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-83bc75a5-ae8b-41aa-bace-d198fa9896a4 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:15.692410 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-83bc75a5-ae8b-41aa-bace-d198fa9896a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:15.697535 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a872f-c7dd-4da4-aa09-cbe5aaee3de2 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:15.699906 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a872f-c7dd-4da4-aa09-cbe5aaee3de2 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:15.699906 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a872f-c7dd-4da4-aa09-cbe5aaee3de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:15.711250 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 920/1866] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:15.712787 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc75a5-ae8b-41aa-bace-d198fa9896a4 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:15.719118 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-892a872f-c7dd-4da4-aa09-cbe5aaee3de2 None None] MFA Rules not processed for user `990f5837994f4a088c656c9b33e9a0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:15.720473 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-892a872f-c7dd-4da4-aa09-cbe5aaee3de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:15.724179 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc75a5-ae8b-41aa-bace-d198fa9896a4 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:15.729558 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5671ede1-c714-437a-9816-5c509b86523b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:15.730256 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5671ede1-c714-437a-9816-5c509b86523b admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:15.730600 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5671ede1-c714-437a-9816-5c509b86523b admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:15.731003 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5671ede1-c714-437a-9816-5c509b86523b admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:15.731792 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5671ede1-c714-437a-9816-5c509b86523b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:50:15.732159 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5671ede1-c714-437a-9816-5c509b86523b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:15.734335 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5671ede1-c714-437a-9816-5c509b86523b admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:15.738264 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:15.761025 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 921/1867] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:50:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:50:15.766369 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5671ede1-c714-437a-9816-5c509b86523b admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:15.781068 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5671ede1-c714-437a-9816-5c509b86523b admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:15.801839 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b50dd4-15bc-4511-9a0e-49dae4a25569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:15.803221 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b50dd4-15bc-4511-9a0e-49dae4a25569 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:15.804537 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b50dd4-15bc-4511-9a0e-49dae4a25569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:15.804818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b50dd4-15bc-4511-9a0e-49dae4a25569 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:15.805458 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b50dd4-15bc-4511-9a0e-49dae4a25569 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:15.807482 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b50dd4-15bc-4511-9a0e-49dae4a25569 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:15.817025 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 947/1868] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Oct 3 12:50:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:15.889452 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a872f-c7dd-4da4-aa09-cbe5aaee3de2 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:15.891507 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-892a872f-c7dd-4da4-aa09-cbe5aaee3de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:15.907585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 922/1869] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:50:15 2022] POST /identity/v3/projects => generated 379 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:15.912280 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b50dd4-15bc-4511-9a0e-49dae4a25569 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:15.925835 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a872f-c7dd-4da4-aa09-cbe5aaee3de2 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:16.014825 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb123a9-dab4-4929-b759-2bad1ec7ce66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:16.015785 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb123a9-dab4-4929-b759-2bad1ec7ce66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:16.016135 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb123a9-dab4-4929-b759-2bad1ec7ce66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:16.016470 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb123a9-dab4-4929-b759-2bad1ec7ce66 admin admin] PATH_INFO: `/v3/users/97b2a559b7b64db09e8db8b163ba3704` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:16.018489 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb123a9-dab4-4929-b759-2bad1ec7ce66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97b2a559b7b64db09e8db8b163ba3704)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:16.019527 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b50dd4-15bc-4511-9a0e-49dae4a25569 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:16.020993 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb123a9-dab4-4929-b759-2bad1ec7ce66 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:16.028183 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7681ed-c93c-4cd5-bfac-fc000b3d8b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:16.030053 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7681ed-c93c-4cd5-bfac-fc000b3d8b0b admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:16.030549 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7681ed-c93c-4cd5-bfac-fc000b3d8b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:16.031041 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7681ed-c93c-4cd5-bfac-fc000b3d8b0b admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:16.031895 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7681ed-c93c-4cd5-bfac-fc000b3d8b0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:16.036098 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7681ed-c93c-4cd5-bfac-fc000b3d8b0b admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:16.145928 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 923/1870] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:50:15 2022] POST /identity/v3/users => generated 290 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:16.146878 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7681ed-c93c-4cd5-bfac-fc000b3d8b0b admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:16.161517 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7681ed-c93c-4cd5-bfac-fc000b3d8b0b admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:16.166775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e87fcdb-c871-4171-8d89-dbf9ce7536ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:16.168647 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e87fcdb-c871-4171-8d89-dbf9ce7536ff admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:16.168647 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e87fcdb-c871-4171-8d89-dbf9ce7536ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:16.168647 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e87fcdb-c871-4171-8d89-dbf9ce7536ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:16.169722 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e87fcdb-c871-4171-8d89-dbf9ce7536ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:16.172061 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e87fcdb-c871-4171-8d89-dbf9ce7536ff admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:16.185231 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 924/1871] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:50:16.195113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e87fcdb-c871-4171-8d89-dbf9ce7536ff admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:16.197396 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1e87fcdb-c871-4171-8d89-dbf9ce7536ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:16.280530 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-1eb123a9-dab4-4929-b759-2bad1ec7ce66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97b2a559b7b64db09e8db8b163ba3704'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:16.298759 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-1eb123a9-dab4-4929-b759-2bad1ec7ce66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97b2a559b7b64db09e8db8b163ba3704'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:16.327449 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-1eb123a9-dab4-4929-b759-2bad1ec7ce66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97b2a559b7b64db09e8db8b163ba3704'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:16.328952 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-1eb123a9-dab4-4929-b759-2bad1ec7ce66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97b2a559b7b64db09e8db8b163ba3704'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:16.349659 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e87fcdb-c871-4171-8d89-dbf9ce7536ff admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:16.358161 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e4fc6-adba-4adc-8714-52aadc6f16b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:16.358884 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e4fc6-adba-4adc-8714-52aadc6f16b1 service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:16.359392 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e4fc6-adba-4adc-8714-52aadc6f16b1 service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:16.359746 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e4fc6-adba-4adc-8714-52aadc6f16b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:16.362220 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b11e4fc6-adba-4adc-8714-52aadc6f16b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:16.376381 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11e4fc6-adba-4adc-8714-52aadc6f16b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:16.378765 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11e4fc6-adba-4adc-8714-52aadc6f16b1 service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:16.391626 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 925/1872] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:50:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:50:16.393012 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e4fc6-adba-4adc-8714-52aadc6f16b1 service nova] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:16.394332 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b11e4fc6-adba-4adc-8714-52aadc6f16b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:16.412735 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e4fc6-adba-4adc-8714-52aadc6f16b1 service nova] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:16.418671 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234471d-10da-4692-bc7f-fb5c1c37f802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:16.419540 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234471d-10da-4692-bc7f-fb5c1c37f802 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:16.420007 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234471d-10da-4692-bc7f-fb5c1c37f802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:16.420461 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234471d-10da-4692-bc7f-fb5c1c37f802 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:16.421282 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8234471d-10da-4692-bc7f-fb5c1c37f802 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:16.423397 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8234471d-10da-4692-bc7f-fb5c1c37f802 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:16.441928 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 926/1873] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:16 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:50:16.443043 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234471d-10da-4692-bc7f-fb5c1c37f802 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:16.459686 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-1eb123a9-dab4-4929-b759-2bad1ec7ce66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97b2a559b7b64db09e8db8b163ba3704'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:16.473461 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234471d-10da-4692-bc7f-fb5c1c37f802 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:16.484126 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 948/1874] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:50:15 2022] DELETE /identity/v3/users/97b2a559b7b64db09e8db8b163ba3704 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:50:16.488972 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f8ee48-db1d-4d7e-a2c5-8a6df5ded50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:16.490213 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8ee48-db1d-4d7e-a2c5-8a6df5ded50c admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:16.490507 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8ee48-db1d-4d7e-a2c5-8a6df5ded50c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:16.492790 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8ee48-db1d-4d7e-a2c5-8a6df5ded50c admin admin] PATH_INFO: `/v3/projects/98c9f163ead84b2d8c847da28f3033d7/users/f3f50a7accc04668bdf977adfdd82bf0/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:16.493052 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a1629-4743-46f4-94ea-6b7a4e221e13 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:16.493052 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a1629-4743-46f4-94ea-6b7a4e221e13 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:16.493645 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a1629-4743-46f4-94ea-6b7a4e221e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:16.512428 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f8ee48-db1d-4d7e-a2c5-8a6df5ded50c admin admin] RBAC: Authorizing `identity:create_grant(project_id=98c9f163ead84b2d8c847da28f3033d7, user_id=f3f50a7accc04668bdf977adfdd82bf0, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:16.515134 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f8ee48-db1d-4d7e-a2c5-8a6df5ded50c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:16.531078 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-633a1629-4743-46f4-94ea-6b7a4e221e13 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:16.532125 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-633a1629-4743-46f4-94ea-6b7a4e221e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:16.607421 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 949/1875] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:16.619421 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a1629-4743-46f4-94ea-6b7a4e221e13 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:16.620858 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-633a1629-4743-46f4-94ea-6b7a4e221e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:16.679160 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 927/1876] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:50:16 2022] PUT /identity/v3/projects/98c9f163ead84b2d8c847da28f3033d7/users/f3f50a7accc04668bdf977adfdd82bf0/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:16.691196 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828c700-941b-456a-8e5f-265e72b7bba4 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:16.691600 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828c700-941b-456a-8e5f-265e72b7bba4 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:16.691965 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828c700-941b-456a-8e5f-265e72b7bba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:16.698441 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a1629-4743-46f4-94ea-6b7a4e221e13 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:16.740882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-c828c700-941b-456a-8e5f-265e72b7bba4 None None] MFA Rules not processed for user `f3f50a7accc04668bdf977adfdd82bf0`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:16.742517 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-c828c700-941b-456a-8e5f-265e72b7bba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:16.789011 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d2b509-bbf4-403b-8c13-b0331270d4b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:16.789011 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2b509-bbf4-403b-8c13-b0331270d4b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:16.789011 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2b509-bbf4-403b-8c13-b0331270d4b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:16.789340 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2b509-bbf4-403b-8c13-b0331270d4b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:16.838707 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 928/1877] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:50:16.843100 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c828c700-941b-456a-8e5f-265e72b7bba4 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:16.874353 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d2b509-bbf4-403b-8c13-b0331270d4b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:16.875834 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d2b509-bbf4-403b-8c13-b0331270d4b4 service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:16.888159 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 950/1878] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:50:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:50:16.908559 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c828c700-941b-456a-8e5f-265e72b7bba4 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:16.915569 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed559b4-59ad-4782-807d-ee8db88c3eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:16.919592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed559b4-59ad-4782-807d-ee8db88c3eab admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:16.919592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed559b4-59ad-4782-807d-ee8db88c3eab admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:16.919592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed559b4-59ad-4782-807d-ee8db88c3eab admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:16.919592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed559b4-59ad-4782-807d-ee8db88c3eab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:16.919922 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed559b4-59ad-4782-807d-ee8db88c3eab admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:17.032422 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 929/1879] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:50:16 2022] POST /identity/v3/projects => generated 379 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:50:17.036751 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d2b509-bbf4-403b-8c13-b0331270d4b4 service neutron] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:17.046312 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d2b509-bbf4-403b-8c13-b0331270d4b4 service neutron] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:17.104022 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16c2ab5-dcf5-4113-a066-12d9de83fabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:17.104642 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c2ab5-dcf5-4113-a066-12d9de83fabe admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:17.104922 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c2ab5-dcf5-4113-a066-12d9de83fabe admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:17.105194 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c2ab5-dcf5-4113-a066-12d9de83fabe admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:17.105832 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16c2ab5-dcf5-4113-a066-12d9de83fabe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:17.107806 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16c2ab5-dcf5-4113-a066-12d9de83fabe admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:17.181745 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 951/1880] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:50:17 2022] POST /identity/v3/users => generated 290 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:17.182918 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed559b4-59ad-4782-807d-ee8db88c3eab admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:17.194463 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed559b4-59ad-4782-807d-ee8db88c3eab admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:17.199406 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026b58c3-df95-4642-9e48-bbd2299bc017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:17.200028 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b58c3-df95-4642-9e48-bbd2299bc017 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:17.200331 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b58c3-df95-4642-9e48-bbd2299bc017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:17.200604 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b58c3-df95-4642-9e48-bbd2299bc017 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:17.201248 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026b58c3-df95-4642-9e48-bbd2299bc017 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:17.202830 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026b58c3-df95-4642-9e48-bbd2299bc017 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:17.215362 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 930/1881] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 12:50:17.219260 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16c2ab5-dcf5-4113-a066-12d9de83fabe admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:17.231268 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16c2ab5-dcf5-4113-a066-12d9de83fabe admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:17.238560 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:17.241341 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff4878c-a8ec-48af-8f8d-6dfd540ca73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:17.242267 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff4878c-a8ec-48af-8f8d-6dfd540ca73e admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:17.242418 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff4878c-a8ec-48af-8f8d-6dfd540ca73e admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:17.242725 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff4878c-a8ec-48af-8f8d-6dfd540ca73e admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:17.243417 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff4878c-a8ec-48af-8f8d-6dfd540ca73e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:17.246015 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff4878c-a8ec-48af-8f8d-6dfd540ca73e admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:17.261020 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 952/1882] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:17 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:50:17.265331 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026b58c3-df95-4642-9e48-bbd2299bc017 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:17.276790 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026b58c3-df95-4642-9e48-bbd2299bc017 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:17.282253 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170dd9ca-c4f6-42fd-a911-eab67e21829c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:17.285707 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170dd9ca-c4f6-42fd-a911-eab67e21829c admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:17.285707 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170dd9ca-c4f6-42fd-a911-eab67e21829c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:17.285707 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170dd9ca-c4f6-42fd-a911-eab67e21829c admin admin] PATH_INFO: `/v3/projects/65b666bb7b63475fbff28a9d208db1e1/users/43dc67ea5ca74884871c43e90288372b/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:17.309131 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170dd9ca-c4f6-42fd-a911-eab67e21829c admin admin] RBAC: Authorizing `identity:create_grant(project_id=65b666bb7b63475fbff28a9d208db1e1, user_id=43dc67ea5ca74884871c43e90288372b, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:17.311820 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170dd9ca-c4f6-42fd-a911-eab67e21829c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:17.394873 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 931/1883] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:50:17 2022] PUT /identity/v3/projects/65b666bb7b63475fbff28a9d208db1e1/users/43dc67ea5ca74884871c43e90288372b/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:17.405211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d71db0-9f0e-465a-baec-af2240845786 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:17.405504 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d71db0-9f0e-465a-baec-af2240845786 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:17.405813 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d71db0-9f0e-465a-baec-af2240845786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:17.436386 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-82d71db0-9f0e-465a-baec-af2240845786 None None] MFA Rules not processed for user `43dc67ea5ca74884871c43e90288372b`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:17.437582 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-82d71db0-9f0e-465a-baec-af2240845786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:17.479220 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170dd9ca-c4f6-42fd-a911-eab67e21829c admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:17.481137 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-170dd9ca-c4f6-42fd-a911-eab67e21829c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:17.515409 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 953/1884] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:17.520613 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d71db0-9f0e-465a-baec-af2240845786 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:17.565078 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170dd9ca-c4f6-42fd-a911-eab67e21829c admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:17.569908 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f50ae4-334e-467a-8e7c-1f13eb4a7335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:17.570543 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f50ae4-334e-467a-8e7c-1f13eb4a7335 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:17.570823 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f50ae4-334e-467a-8e7c-1f13eb4a7335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:17.571092 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f50ae4-334e-467a-8e7c-1f13eb4a7335 admin admin] PATH_INFO: `/v3/projects/f04332e6d0ce41ca823ea6c2c7e47ed7` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:17.572220 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f50ae4-334e-467a-8e7c-1f13eb4a7335 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f04332e6d0ce41ca823ea6c2c7e47ed7)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:17.574190 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f50ae4-334e-467a-8e7c-1f13eb4a7335 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:17.591284 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d71db0-9f0e-465a-baec-af2240845786 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:17.597183 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357ab74e-55dc-40f0-b5f8-1d9a5961744a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:17.598119 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ab74e-55dc-40f0-b5f8-1d9a5961744a admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:17.598119 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ab74e-55dc-40f0-b5f8-1d9a5961744a admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:17.598463 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ab74e-55dc-40f0-b5f8-1d9a5961744a admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:17.598938 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357ab74e-55dc-40f0-b5f8-1d9a5961744a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:17.600873 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357ab74e-55dc-40f0-b5f8-1d9a5961744a admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:17.745961 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-20f50ae4-334e-467a-8e7c-1f13eb4a7335 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f04332e6d0ce41ca823ea6c2c7e47ed7', 'request_id': 'req-20f50ae4-334e-467a-8e7c-1f13eb4a7335'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:17.751447 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-20f50ae4-334e-467a-8e7c-1f13eb4a7335 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f04332e6d0ce41ca823ea6c2c7e47ed7', 'request_id': 'req-20f50ae4-334e-467a-8e7c-1f13eb4a7335'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:17.812925 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 954/1885] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:50:17 2022] POST /identity/v3/projects => generated 377 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:17.814220 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357ab74e-55dc-40f0-b5f8-1d9a5961744a admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:17.865657 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-20f50ae4-334e-467a-8e7c-1f13eb4a7335 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f04332e6d0ce41ca823ea6c2c7e47ed7', 'request_id': 'req-20f50ae4-334e-467a-8e7c-1f13eb4a7335'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:17.880125 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-20f50ae4-334e-467a-8e7c-1f13eb4a7335 admin admin] The token cache is being invalidate because project f04332e6d0ce41ca823ea6c2c7e47ed7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:50:17.880630 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-20f50ae4-334e-467a-8e7c-1f13eb4a7335 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:17.881540 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-20f50ae4-334e-467a-8e7c-1f13eb4a7335 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:17.883386 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 932/1886] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:50:17 2022] DELETE /identity/v3/projects/f04332e6d0ce41ca823ea6c2c7e47ed7 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:50:17.886513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357ab74e-55dc-40f0-b5f8-1d9a5961744a admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:17.887374 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f50ae4-334e-467a-8e7c-1f13eb4a7335 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:17.888976 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-20f50ae4-334e-467a-8e7c-1f13eb4a7335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:17.893426 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d89519f-12bf-463a-9b5b-2c893c6054f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:17.894070 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d89519f-12bf-463a-9b5b-2c893c6054f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:17.894339 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d89519f-12bf-463a-9b5b-2c893c6054f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:17.894612 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d89519f-12bf-463a-9b5b-2c893c6054f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:17.895246 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d89519f-12bf-463a-9b5b-2c893c6054f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:17.897205 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d89519f-12bf-463a-9b5b-2c893c6054f8 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:17.956579 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f50ae4-334e-467a-8e7c-1f13eb4a7335 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:17.961337 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c058277e-c31e-4a38-a5f2-63c13b383b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:17.962030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058277e-c31e-4a38-a5f2-63c13b383b0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:17.962310 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058277e-c31e-4a38-a5f2-63c13b383b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:17.962612 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058277e-c31e-4a38-a5f2-63c13b383b0b admin admin] PATH_INFO: `/v3/users/5a0c6feb988a4408946ee2de8096837f` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:17.963749 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c058277e-c31e-4a38-a5f2-63c13b383b0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a0c6feb988a4408946ee2de8096837f)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:17.965653 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c058277e-c31e-4a38-a5f2-63c13b383b0b admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:18.007192 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 955/1887] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:50:17 2022] POST /identity/v3/users => generated 295 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:18.008374 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d89519f-12bf-463a-9b5b-2c893c6054f8 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:18.009974 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-0d89519f-12bf-463a-9b5b-2c893c6054f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:18.037185 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d89519f-12bf-463a-9b5b-2c893c6054f8 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:18.041892 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1498af6-1843-43d0-957e-d1ff63eb1c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:18.042574 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1498af6-1843-43d0-957e-d1ff63eb1c1c admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:18.042907 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1498af6-1843-43d0-957e-d1ff63eb1c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:18.043171 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1498af6-1843-43d0-957e-d1ff63eb1c1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:18.043838 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1498af6-1843-43d0-957e-d1ff63eb1c1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:18.045406 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1498af6-1843-43d0-957e-d1ff63eb1c1c admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:18.055303 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 956/1888] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:50:18.059541 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1498af6-1843-43d0-957e-d1ff63eb1c1c admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:18.069931 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1498af6-1843-43d0-957e-d1ff63eb1c1c admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:18.072314 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:18.076770 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8fb86c-0cb6-40d6-881c-9f0b19f02d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:18.077414 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8fb86c-0cb6-40d6-881c-9f0b19f02d95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:18.077698 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8fb86c-0cb6-40d6-881c-9f0b19f02d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:18.077976 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8fb86c-0cb6-40d6-881c-9f0b19f02d95 admin admin] PATH_INFO: `/v3/projects/c491ea2cbe9248739252f4e9ad881125/users/5de683d15f7849ab98ae0d608ca1059b/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:18.093925 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8fb86c-0cb6-40d6-881c-9f0b19f02d95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c491ea2cbe9248739252f4e9ad881125, user_id=5de683d15f7849ab98ae0d608ca1059b, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:18.097135 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8fb86c-0cb6-40d6-881c-9f0b19f02d95 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:18.168230 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 957/1889] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:50:18 2022] PUT /identity/v3/projects/c491ea2cbe9248739252f4e9ad881125/users/5de683d15f7849ab98ae0d608ca1059b/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:50:18.172248 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8fb86c-0cb6-40d6-881c-9f0b19f02d95 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:18.178833 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c058277e-c31e-4a38-a5f2-63c13b383b0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a0c6feb988a4408946ee2de8096837f'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:18.192134 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c058277e-c31e-4a38-a5f2-63c13b383b0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a0c6feb988a4408946ee2de8096837f'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:18.192913 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c058277e-c31e-4a38-a5f2-63c13b383b0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a0c6feb988a4408946ee2de8096837f'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:18.199916 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c058277e-c31e-4a38-a5f2-63c13b383b0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a0c6feb988a4408946ee2de8096837f'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:18.233937 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8fb86c-0cb6-40d6-881c-9f0b19f02d95 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:18.239453 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a345b7-42ff-4039-8a57-1f19d90743fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:18.240091 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a345b7-42ff-4039-8a57-1f19d90743fd admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:18.240421 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a345b7-42ff-4039-8a57-1f19d90743fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:18.240695 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a345b7-42ff-4039-8a57-1f19d90743fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:18.241333 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a345b7-42ff-4039-8a57-1f19d90743fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:18.242918 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a345b7-42ff-4039-8a57-1f19d90743fd admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:18.253817 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 958/1890] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 12:50:18.258262 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a345b7-42ff-4039-8a57-1f19d90743fd admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:18.259896 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-c7a345b7-42ff-4039-8a57-1f19d90743fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:18.272973 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a345b7-42ff-4039-8a57-1f19d90743fd admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:18.277712 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f9981-fc49-4255-9458-7de9926ba32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:18.278361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f9981-fc49-4255-9458-7de9926ba32a admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:18.278643 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f9981-fc49-4255-9458-7de9926ba32a admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:18.278918 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f9981-fc49-4255-9458-7de9926ba32a admin admin] PATH_INFO: `/v3/projects/c491ea2cbe9248739252f4e9ad881125/users/5de683d15f7849ab98ae0d608ca1059b/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:18.280867 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9f9981-fc49-4255-9458-7de9926ba32a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c491ea2cbe9248739252f4e9ad881125, user_id=5de683d15f7849ab98ae0d608ca1059b, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:18.283805 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9f9981-fc49-4255-9458-7de9926ba32a admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:18.288044 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c058277e-c31e-4a38-a5f2-63c13b383b0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a0c6feb988a4408946ee2de8096837f'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:18.294022 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 959/1891] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:50:18 2022] PUT /identity/v3/projects/c491ea2cbe9248739252f4e9ad881125/users/5de683d15f7849ab98ae0d608ca1059b/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:50:18.298013 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 933/1892] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:50:17 2022] DELETE /identity/v3/users/5a0c6feb988a4408946ee2de8096837f => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:18.305436 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c15ac-d966-487f-98b7-51dc7881e8b2 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:18.305755 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c15ac-d966-487f-98b7-51dc7881e8b2 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:18.305996 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c15ac-d966-487f-98b7-51dc7881e8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:18.341224 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-da5c15ac-d966-487f-98b7-51dc7881e8b2 None None] MFA Rules not processed for user `5de683d15f7849ab98ae0d608ca1059b`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:18.342443 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-da5c15ac-d966-487f-98b7-51dc7881e8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:18.403339 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:18.418930 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 960/1893] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:18 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:18.430694 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c058277e-c31e-4a38-a5f2-63c13b383b0b admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:18.432130 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-c058277e-c31e-4a38-a5f2-63c13b383b0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:18.498313 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c058277e-c31e-4a38-a5f2-63c13b383b0b admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:18.502980 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81278605-55ae-4ced-a9dc-0e67eb94515e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:18.503613 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81278605-55ae-4ced-a9dc-0e67eb94515e service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:18.503899 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81278605-55ae-4ced-a9dc-0e67eb94515e service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:18.504139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81278605-55ae-4ced-a9dc-0e67eb94515e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:18.514052 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81278605-55ae-4ced-a9dc-0e67eb94515e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:18.515522 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81278605-55ae-4ced-a9dc-0e67eb94515e service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:18.527531 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 934/1894] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:50:18 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:50:18.672830 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5c15ac-d966-487f-98b7-51dc7881e8b2 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:18.686169 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5c15ac-d966-487f-98b7-51dc7881e8b2 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:18.691621 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0020aa54-b0c4-4bc2-a0b0-0ca34330bed4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:18.692754 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020aa54-b0c4-4bc2-a0b0-0ca34330bed4 service cinder] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:18.693211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020aa54-b0c4-4bc2-a0b0-0ca34330bed4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:18.693656 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020aa54-b0c4-4bc2-a0b0-0ca34330bed4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:18.695801 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-0020aa54-b0c4-4bc2-a0b0-0ca34330bed4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:18.775448 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0020aa54-b0c4-4bc2-a0b0-0ca34330bed4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:18.777172 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0020aa54-b0c4-4bc2-a0b0-0ca34330bed4 service cinder] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:18.789603 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 961/1895] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:50:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:50:18.884663 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81278605-55ae-4ced-a9dc-0e67eb94515e service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:18.886184 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-81278605-55ae-4ced-a9dc-0e67eb94515e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:18.962254 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81278605-55ae-4ced-a9dc-0e67eb94515e service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:18.967385 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66162720-0df7-4c9e-99d9-1af42c61b81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:18.968164 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66162720-0df7-4c9e-99d9-1af42c61b81b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:18.968510 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66162720-0df7-4c9e-99d9-1af42c61b81b admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:18.968847 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66162720-0df7-4c9e-99d9-1af42c61b81b admin admin] PATH_INFO: `/v3/projects/a2da15bfe6274a6cab836f1786a847fe` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:18.970197 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66162720-0df7-4c9e-99d9-1af42c61b81b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2da15bfe6274a6cab836f1786a847fe)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:18.973104 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66162720-0df7-4c9e-99d9-1af42c61b81b admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:19.096861 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-66162720-0df7-4c9e-99d9-1af42c61b81b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2da15bfe6274a6cab836f1786a847fe', 'request_id': 'req-66162720-0df7-4c9e-99d9-1af42c61b81b'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:19.104501 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-66162720-0df7-4c9e-99d9-1af42c61b81b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2da15bfe6274a6cab836f1786a847fe', 'request_id': 'req-66162720-0df7-4c9e-99d9-1af42c61b81b'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:19.204005 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-66162720-0df7-4c9e-99d9-1af42c61b81b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2da15bfe6274a6cab836f1786a847fe', 'request_id': 'req-66162720-0df7-4c9e-99d9-1af42c61b81b'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:19.221392 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-66162720-0df7-4c9e-99d9-1af42c61b81b admin admin] The token cache is being invalidate because project a2da15bfe6274a6cab836f1786a847fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:50:19.221829 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-66162720-0df7-4c9e-99d9-1af42c61b81b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:19.222595 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-66162720-0df7-4c9e-99d9-1af42c61b81b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:19.224325 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 935/1896] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:50:18 2022] DELETE /identity/v3/projects/a2da15bfe6274a6cab836f1786a847fe => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:19.230417 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e32b0d-4eba-4efe-a365-3ca8abd8ba66 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:19.230919 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e32b0d-4eba-4efe-a365-3ca8abd8ba66 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:19.231027 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e32b0d-4eba-4efe-a365-3ca8abd8ba66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:19.255785 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-51e32b0d-4eba-4efe-a365-3ca8abd8ba66 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:19.257334 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-51e32b0d-4eba-4efe-a365-3ca8abd8ba66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:19.331896 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 962/1897] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:50:19.348246 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa16bc-137d-4aa4-b41b-15fea2ad812a None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:19.348545 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa16bc-137d-4aa4-b41b-15fea2ad812a None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:19.348831 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa16bc-137d-4aa4-b41b-15fea2ad812a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:19.370110 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-caaa16bc-137d-4aa4-b41b-15fea2ad812a None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:19.371350 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-caaa16bc-137d-4aa4-b41b-15fea2ad812a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:19.380473 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 936/1898] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:19.384944 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e32b0d-4eba-4efe-a365-3ca8abd8ba66 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:19.395008 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e32b0d-4eba-4efe-a365-3ca8abd8ba66 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:19.399655 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca41209-de8e-4e60-af11-dd6e08851fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:19.400252 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca41209-de8e-4e60-af11-dd6e08851fc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:19.400528 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca41209-de8e-4e60-af11-dd6e08851fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:19.400797 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca41209-de8e-4e60-af11-dd6e08851fc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:19.401470 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca41209-de8e-4e60-af11-dd6e08851fc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:50:19.401769 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca41209-de8e-4e60-af11-dd6e08851fc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:19.403385 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca41209-de8e-4e60-af11-dd6e08851fc9 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:19.415744 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 963/1899] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:50:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:50:19.420107 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa16bc-137d-4aa4-b41b-15fea2ad812a None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:19.431602 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa16bc-137d-4aa4-b41b-15fea2ad812a None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:19.436448 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c0cff8-e46e-451d-8a39-496162103087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:19.437731 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0cff8-e46e-451d-8a39-496162103087 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:19.437731 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0cff8-e46e-451d-8a39-496162103087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:19.437731 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0cff8-e46e-451d-8a39-496162103087 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:19.438363 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c0cff8-e46e-451d-8a39-496162103087 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:19.440839 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c0cff8-e46e-451d-8a39-496162103087 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:19.563880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 937/1900] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:50:19 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:19.568148 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca41209-de8e-4e60-af11-dd6e08851fc9 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:19.578366 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca41209-de8e-4e60-af11-dd6e08851fc9 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:19.649453 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b18eb59-ca37-40b1-9efb-33196edf0d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:19.650169 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18eb59-ca37-40b1-9efb-33196edf0d47 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:19.650524 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18eb59-ca37-40b1-9efb-33196edf0d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:19.650858 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18eb59-ca37-40b1-9efb-33196edf0d47 admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:19.651700 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b18eb59-ca37-40b1-9efb-33196edf0d47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:19.654083 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b18eb59-ca37-40b1-9efb-33196edf0d47 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:19.781204 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 964/1901] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:50:19 2022] POST /identity/v3/users => generated 293 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:19.786160 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c0cff8-e46e-451d-8a39-496162103087 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:19.801368 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c0cff8-e46e-451d-8a39-496162103087 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:19.809866 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cc4ac5-039f-41bf-ae6e-5fbe20b93aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:19.809866 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cc4ac5-039f-41bf-ae6e-5fbe20b93aab admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:19.809866 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cc4ac5-039f-41bf-ae6e-5fbe20b93aab admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:19.809866 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cc4ac5-039f-41bf-ae6e-5fbe20b93aab admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:19.810187 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cc4ac5-039f-41bf-ae6e-5fbe20b93aab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:19.811059 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cc4ac5-039f-41bf-ae6e-5fbe20b93aab admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:19.824667 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 938/1902] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:19 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:50:19.828514 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b18eb59-ca37-40b1-9efb-33196edf0d47 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:19.839997 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b18eb59-ca37-40b1-9efb-33196edf0d47 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:19.845177 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbe33d-74b3-4d79-9ef7-2edd6b059255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:19.845797 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbe33d-74b3-4d79-9ef7-2edd6b059255 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:19.846079 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbe33d-74b3-4d79-9ef7-2edd6b059255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:19.846351 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbe33d-74b3-4d79-9ef7-2edd6b059255 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:19.851720 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fbe33d-74b3-4d79-9ef7-2edd6b059255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:19.854468 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fbe33d-74b3-4d79-9ef7-2edd6b059255 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:19.869377 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 965/1903] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:19 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 12:50:19.873986 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cc4ac5-039f-41bf-ae6e-5fbe20b93aab admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:19.889377 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cc4ac5-039f-41bf-ae6e-5fbe20b93aab admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:19.895698 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea60291-2b53-43a0-b950-f22d8d400cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:19.896390 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea60291-2b53-43a0-b950-f22d8d400cbf admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:19.896848 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea60291-2b53-43a0-b950-f22d8d400cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:19.897123 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea60291-2b53-43a0-b950-f22d8d400cbf admin admin] PATH_INFO: `/v3/projects/4ca252947c74475d98ede7ec92ecc9c5/users/c46900e3bb3147a7840a76bef9268bf0/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:19.908827 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:19.918961 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea60291-2b53-43a0-b950-f22d8d400cbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ca252947c74475d98ede7ec92ecc9c5, user_id=c46900e3bb3147a7840a76bef9268bf0, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:19.922998 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea60291-2b53-43a0-b950-f22d8d400cbf admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:19.986236 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 939/1904] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:50:19 2022] PUT /identity/v3/projects/4ca252947c74475d98ede7ec92ecc9c5/users/c46900e3bb3147a7840a76bef9268bf0/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:19.998540 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0978669-e31a-410f-8b5d-ee0014c56c54 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:19.998823 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0978669-e31a-410f-8b5d-ee0014c56c54 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:19.999097 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0978669-e31a-410f-8b5d-ee0014c56c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:20.040841 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-d0978669-e31a-410f-8b5d-ee0014c56c54 None None] MFA Rules not processed for user `c46900e3bb3147a7840a76bef9268bf0`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:20.042125 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-d0978669-e31a-410f-8b5d-ee0014c56c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:20.080945 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:20.146851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 966/1905] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:20.153312 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea60291-2b53-43a0-b950-f22d8d400cbf admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:20.227080 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea60291-2b53-43a0-b950-f22d8d400cbf admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:20.236138 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a7681-775c-49ad-ad98-522059f3d819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:20.236138 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a7681-775c-49ad-ad98-522059f3d819 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:20.236138 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a7681-775c-49ad-ad98-522059f3d819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:20.236138 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a7681-775c-49ad-ad98-522059f3d819 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:20.236854 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1a7681-775c-49ad-ad98-522059f3d819 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:20.239566 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1a7681-775c-49ad-ad98-522059f3d819 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:20.379435 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 940/1906] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:50:20 2022] POST /identity/v3/projects => generated 385 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:20.379990 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0978669-e31a-410f-8b5d-ee0014c56c54 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:20.401084 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0978669-e31a-410f-8b5d-ee0014c56c54 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:20.486345 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c5eac-d462-4f95-8c20-6903e9bece58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:20.487110 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c5eac-d462-4f95-8c20-6903e9bece58 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:20.488614 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c5eac-d462-4f95-8c20-6903e9bece58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:20.488886 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c5eac-d462-4f95-8c20-6903e9bece58 admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:20.489880 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c5eac-d462-4f95-8c20-6903e9bece58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:20.492413 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c5eac-d462-4f95-8c20-6903e9bece58 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:20.609035 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 967/1907] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:50:20 2022] POST /identity/v3/users => generated 293 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:20.613281 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a7681-775c-49ad-ad98-522059f3d819 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:20.637388 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a7681-775c-49ad-ad98-522059f3d819 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:20.643861 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c66ca-24d2-455e-aca7-7e1a329faf60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:20.645251 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c66ca-24d2-455e-aca7-7e1a329faf60 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:20.645540 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c66ca-24d2-455e-aca7-7e1a329faf60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:20.645810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c66ca-24d2-455e-aca7-7e1a329faf60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:20.646990 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c66ca-24d2-455e-aca7-7e1a329faf60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:20.648889 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c66ca-24d2-455e-aca7-7e1a329faf60 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:20.667666 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 941/1908] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:20 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:50:20.669374 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c5eac-d462-4f95-8c20-6903e9bece58 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:20.693967 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c5eac-d462-4f95-8c20-6903e9bece58 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:20.720981 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f6cc6b-8b72-4d5e-85dd-0dfa4f46cd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:20.721997 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6cc6b-8b72-4d5e-85dd-0dfa4f46cd33 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:20.722359 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6cc6b-8b72-4d5e-85dd-0dfa4f46cd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:20.724008 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6cc6b-8b72-4d5e-85dd-0dfa4f46cd33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:20.724940 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f6cc6b-8b72-4d5e-85dd-0dfa4f46cd33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:20.729026 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f6cc6b-8b72-4d5e-85dd-0dfa4f46cd33 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:20.749192 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 968/1909] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:20 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:50:20.749847 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c66ca-24d2-455e-aca7-7e1a329faf60 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:20.761528 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c66ca-24d2-455e-aca7-7e1a329faf60 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:20.767999 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78d4632-2b7a-4ab9-acac-a169930a9d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:20.768848 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d4632-2b7a-4ab9-acac-a169930a9d53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:20.770044 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d4632-2b7a-4ab9-acac-a169930a9d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:20.770459 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d4632-2b7a-4ab9-acac-a169930a9d53 admin admin] PATH_INFO: `/v3/projects/acf8de0832624632906cd4d80337a83c/users/56d217d287a9435896c23e56b762ea4a/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:20.803685 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78d4632-2b7a-4ab9-acac-a169930a9d53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acf8de0832624632906cd4d80337a83c, user_id=56d217d287a9435896c23e56b762ea4a, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:20.822626 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78d4632-2b7a-4ab9-acac-a169930a9d53 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:20.927784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 942/1910] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:50:20 2022] PUT /identity/v3/projects/acf8de0832624632906cd4d80337a83c/users/56d217d287a9435896c23e56b762ea4a/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:20.942758 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd782bfa-11ac-405a-9c5b-d583bd2be665 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:20.943061 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd782bfa-11ac-405a-9c5b-d583bd2be665 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:20.943336 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd782bfa-11ac-405a-9c5b-d583bd2be665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:20.993724 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-bd782bfa-11ac-405a-9c5b-d583bd2be665 None None] MFA Rules not processed for user `56d217d287a9435896c23e56b762ea4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:20.995122 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-bd782bfa-11ac-405a-9c5b-d583bd2be665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:21.064742 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 969/1911] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:21.069592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78d4632-2b7a-4ab9-acac-a169930a9d53 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:21.179523 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78d4632-2b7a-4ab9-acac-a169930a9d53 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:21.186482 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc108b-e97b-47ad-b76e-bf33596f8aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:21.187202 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc108b-e97b-47ad-b76e-bf33596f8aec admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:21.187574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc108b-e97b-47ad-b76e-bf33596f8aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:21.188039 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc108b-e97b-47ad-b76e-bf33596f8aec admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:21.188738 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfc108b-e97b-47ad-b76e-bf33596f8aec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:21.190830 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfc108b-e97b-47ad-b76e-bf33596f8aec admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:21.335519 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 943/1912] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:50:21 2022] POST /identity/v3/projects => generated 385 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:21.339612 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd782bfa-11ac-405a-9c5b-d583bd2be665 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:21.352056 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd782bfa-11ac-405a-9c5b-d583bd2be665 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:21.398400 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:21.435983 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192eb5e6-194c-460c-8937-fcfe442779d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:21.436610 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192eb5e6-194c-460c-8937-fcfe442779d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:21.436891 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192eb5e6-194c-460c-8937-fcfe442779d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:21.437195 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192eb5e6-194c-460c-8937-fcfe442779d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:21.437803 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192eb5e6-194c-460c-8937-fcfe442779d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:21.439763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192eb5e6-194c-460c-8937-fcfe442779d4 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:21.548621 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 970/1913] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:50:21 2022] POST /identity/v3/users => generated 299 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:21.553631 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc108b-e97b-47ad-b76e-bf33596f8aec admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:21.564684 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc108b-e97b-47ad-b76e-bf33596f8aec admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:21.571108 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa16b1e-883c-4181-ab8b-b9a63bf42712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:21.571756 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa16b1e-883c-4181-ab8b-b9a63bf42712 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:21.572035 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa16b1e-883c-4181-ab8b-b9a63bf42712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:21.572306 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa16b1e-883c-4181-ab8b-b9a63bf42712 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:21.572970 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa16b1e-883c-4181-ab8b-b9a63bf42712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:21.574839 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa16b1e-883c-4181-ab8b-b9a63bf42712 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:21.585844 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 944/1914] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:50:21.590502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192eb5e6-194c-460c-8937-fcfe442779d4 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:21.603541 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192eb5e6-194c-460c-8937-fcfe442779d4 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:21.608868 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760809d2-8510-42cb-90e3-c974133a176a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:21.609611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760809d2-8510-42cb-90e3-c974133a176a admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:21.609903 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760809d2-8510-42cb-90e3-c974133a176a admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:21.610187 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760809d2-8510-42cb-90e3-c974133a176a admin admin] PATH_INFO: `/v3/projects/f153d879299443148eddecb8269c117d/users/56215af8bd2b44c8a9509bc4d904c0e5/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:21.627970 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760809d2-8510-42cb-90e3-c974133a176a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f153d879299443148eddecb8269c117d, user_id=56215af8bd2b44c8a9509bc4d904c0e5, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:21.631991 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760809d2-8510-42cb-90e3-c974133a176a admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:21.776204 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 971/1915] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:50:21 2022] PUT /identity/v3/projects/f153d879299443148eddecb8269c117d/users/56215af8bd2b44c8a9509bc4d904c0e5/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:21.777965 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa16b1e-883c-4181-ab8b-b9a63bf42712 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:21.779201 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:21.790894 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa16b1e-883c-4181-ab8b-b9a63bf42712 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:21.852021 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fb574e-b435-46cb-8ba9-c5062ea659a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:21.852642 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb574e-b435-46cb-8ba9-c5062ea659a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:21.852923 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb574e-b435-46cb-8ba9-c5062ea659a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:21.853199 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb574e-b435-46cb-8ba9-c5062ea659a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:21.853831 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fb574e-b435-46cb-8ba9-c5062ea659a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:21.855454 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fb574e-b435-46cb-8ba9-c5062ea659a5 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:21.866259 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 945/1916] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 12:50:21.870635 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760809d2-8510-42cb-90e3-c974133a176a admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:21.880935 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760809d2-8510-42cb-90e3-c974133a176a admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:21.885526 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74996c33-80df-463d-8488-18bc6c23377c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:21.886209 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74996c33-80df-463d-8488-18bc6c23377c admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:21.886530 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74996c33-80df-463d-8488-18bc6c23377c admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:21.886763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74996c33-80df-463d-8488-18bc6c23377c admin admin] PATH_INFO: `/v3/projects/f153d879299443148eddecb8269c117d/users/56215af8bd2b44c8a9509bc4d904c0e5/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:21.888689 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74996c33-80df-463d-8488-18bc6c23377c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f153d879299443148eddecb8269c117d, user_id=56215af8bd2b44c8a9509bc4d904c0e5, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:21.891307 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74996c33-80df-463d-8488-18bc6c23377c admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:21.901426 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 972/1917] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:50:21 2022] PUT /identity/v3/projects/f153d879299443148eddecb8269c117d/users/56215af8bd2b44c8a9509bc4d904c0e5/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:21.913141 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2bcfc8-acf0-4bb8-a2bd-cf96958bfcc6 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:21.913418 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2bcfc8-acf0-4bb8-a2bd-cf96958bfcc6 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:21.913694 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2bcfc8-acf0-4bb8-a2bd-cf96958bfcc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:21.943029 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-6e2bcfc8-acf0-4bb8-a2bd-cf96958bfcc6 None None] MFA Rules not processed for user `56215af8bd2b44c8a9509bc4d904c0e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:21.944371 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-6e2bcfc8-acf0-4bb8-a2bd-cf96958bfcc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:22.024328 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 946/1918] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:21 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:22.036176 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74996c33-80df-463d-8488-18bc6c23377c admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:22.038536 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-74996c33-80df-463d-8488-18bc6c23377c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:22.100355 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74996c33-80df-463d-8488-18bc6c23377c admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:22.103940 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf2eda-0263-409d-aeb2-d9ab1c5eb235 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:22.104553 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf2eda-0263-409d-aeb2-d9ab1c5eb235 service cinder] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:22.104828 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf2eda-0263-409d-aeb2-d9ab1c5eb235 service cinder] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:22.105096 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf2eda-0263-409d-aeb2-d9ab1c5eb235 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:22.169474 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bf2eda-0263-409d-aeb2-d9ab1c5eb235 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:22.170914 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bf2eda-0263-409d-aeb2-d9ab1c5eb235 service cinder] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:22.180955 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 973/1919] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:50:22 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:50:23.266269 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:23.289421 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2bcfc8-acf0-4bb8-a2bd-cf96958bfcc6 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:23.290958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-6e2bcfc8-acf0-4bb8-a2bd-cf96958bfcc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:23.359100 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2bcfc8-acf0-4bb8-a2bd-cf96958bfcc6 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:23.365202 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1291660-4af4-4d0e-9c17-8842e1fd894d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:23.366036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1291660-4af4-4d0e-9c17-8842e1fd894d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:23.366414 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1291660-4af4-4d0e-9c17-8842e1fd894d admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:23.367006 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1291660-4af4-4d0e-9c17-8842e1fd894d admin admin] PATH_INFO: `/v3/users/f3f50a7accc04668bdf977adfdd82bf0` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:23.368301 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1291660-4af4-4d0e-9c17-8842e1fd894d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3f50a7accc04668bdf977adfdd82bf0)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:23.370923 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1291660-4af4-4d0e-9c17-8842e1fd894d admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:23.497540 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:INSERT=3,keystone:DELETE=8,keystone:SELECT=89 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:23.608264 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf2eda-0263-409d-aeb2-d9ab1c5eb235 service cinder] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:23.609788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-b4bf2eda-0263-409d-aeb2-d9ab1c5eb235 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:23.626929 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf2eda-0263-409d-aeb2-d9ab1c5eb235 service cinder] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:23.634922 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1bbfe3-81c6-4f90-9e09-7e75492e4dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:23.636217 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1bbfe3-81c6-4f90-9e09-7e75492e4dac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:23.636683 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1bbfe3-81c6-4f90-9e09-7e75492e4dac admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:23.637107 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1bbfe3-81c6-4f90-9e09-7e75492e4dac admin admin] PATH_INFO: `/v3/users/417b5a24768a4a8991344408f6d003d4` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:23.639232 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1bbfe3-81c6-4f90-9e09-7e75492e4dac admin admin] RBAC: Authorizing `identity:delete_user(user_id=417b5a24768a4a8991344408f6d003d4)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:23.642048 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1bbfe3-81c6-4f90-9e09-7e75492e4dac admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:23.658510 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b1291660-4af4-4d0e-9c17-8842e1fd894d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3f50a7accc04668bdf977adfdd82bf0'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:23.702268 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b1291660-4af4-4d0e-9c17-8842e1fd894d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3f50a7accc04668bdf977adfdd82bf0'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:23.703200 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b1291660-4af4-4d0e-9c17-8842e1fd894d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3f50a7accc04668bdf977adfdd82bf0'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:23.719009 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b1291660-4af4-4d0e-9c17-8842e1fd894d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3f50a7accc04668bdf977adfdd82bf0'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:23.791003 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b1291660-4af4-4d0e-9c17-8842e1fd894d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3f50a7accc04668bdf977adfdd82bf0'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:23.797577 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 947/1920] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:50:23 2022] DELETE /identity/v3/users/f3f50a7accc04668bdf977adfdd82bf0 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:23.802723 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592b3ec-e408-41d1-abc2-ad08f1c16a0f None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:23.803207 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592b3ec-e408-41d1-abc2-ad08f1c16a0f None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:23.803527 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592b3ec-e408-41d1-abc2-ad08f1c16a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:23.823368 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-5592b3ec-e408-41d1-abc2-ad08f1c16a0f None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:23.824759 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-5592b3ec-e408-41d1-abc2-ad08f1c16a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:23.836294 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-5a1bbfe3-81c6-4f90-9e09-7e75492e4dac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '417b5a24768a4a8991344408f6d003d4'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:23.844352 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-5a1bbfe3-81c6-4f90-9e09-7e75492e4dac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '417b5a24768a4a8991344408f6d003d4'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:23.858415 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-5a1bbfe3-81c6-4f90-9e09-7e75492e4dac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '417b5a24768a4a8991344408f6d003d4'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:23.859365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-5a1bbfe3-81c6-4f90-9e09-7e75492e4dac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '417b5a24768a4a8991344408f6d003d4'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:23.892614 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 948/1921] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:23.908373 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5592b3ec-e408-41d1-abc2-ad08f1c16a0f None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:23.909862 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-5592b3ec-e408-41d1-abc2-ad08f1c16a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:23.945817 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-5a1bbfe3-81c6-4f90-9e09-7e75492e4dac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '417b5a24768a4a8991344408f6d003d4'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:23.961165 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 974/1922] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:50:23 2022] DELETE /identity/v3/users/417b5a24768a4a8991344408f6d003d4 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:50:23.963950 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c550c8-061e-4919-84d4-204b90daa276 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:23.964419 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c550c8-061e-4919-84d4-204b90daa276 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:23.964863 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c550c8-061e-4919-84d4-204b90daa276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:23.971904 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5592b3ec-e408-41d1-abc2-ad08f1c16a0f None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:23.998890 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-f7c550c8-061e-4919-84d4-204b90daa276 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:24.000881 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-f7c550c8-061e-4919-84d4-204b90daa276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:24.021831 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:24.030343 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb53489-7cae-4271-8a00-fd9be140d418 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:24.030967 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb53489-7cae-4271-8a00-fd9be140d418 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:24.031247 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb53489-7cae-4271-8a00-fd9be140d418 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:24.031547 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb53489-7cae-4271-8a00-fd9be140d418 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:24.080156 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 975/1923] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:24.091055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c550c8-061e-4919-84d4-204b90daa276 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:24.098502 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb53489-7cae-4271-8a00-fd9be140d418 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:24.100081 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb53489-7cae-4271-8a00-fd9be140d418 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:24.102490 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c550c8-061e-4919-84d4-204b90daa276 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:24.106225 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a6d9b6-d3c2-46eb-8c0a-10c6613c03f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:24.107024 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6d9b6-d3c2-46eb-8c0a-10c6613c03f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:24.107412 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6d9b6-d3c2-46eb-8c0a-10c6613c03f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:24.107750 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6d9b6-d3c2-46eb-8c0a-10c6613c03f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:24.110027 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 949/1924] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:50:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:50:24.118365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a6d9b6-d3c2-46eb-8c0a-10c6613c03f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:24.119832 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a6d9b6-d3c2-46eb-8c0a-10c6613c03f7 service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:24.129669 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 976/1925] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:50:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:50:24.840296 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb53489-7cae-4271-8a00-fd9be140d418 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:24.841285 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ccb53489-7cae-4271-8a00-fd9be140d418 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:24.857881 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb53489-7cae-4271-8a00-fd9be140d418 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:24.862923 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:24.867055 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1551de6b-e2ab-48a5-a9f5-579ff0f5771d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:24.868754 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551de6b-e2ab-48a5-a9f5-579ff0f5771d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:24.868754 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551de6b-e2ab-48a5-a9f5-579ff0f5771d admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:24.869036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551de6b-e2ab-48a5-a9f5-579ff0f5771d admin admin] PATH_INFO: `/v3/projects/98c9f163ead84b2d8c847da28f3033d7` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:24.877610 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1551de6b-e2ab-48a5-a9f5-579ff0f5771d admin admin] RBAC: Authorizing `identity:delete_project(project_id=98c9f163ead84b2d8c847da28f3033d7)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:24.880321 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1551de6b-e2ab-48a5-a9f5-579ff0f5771d admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:24.891918 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:25.016259 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1551de6b-e2ab-48a5-a9f5-579ff0f5771d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98c9f163ead84b2d8c847da28f3033d7', 'request_id': 'req-1551de6b-e2ab-48a5-a9f5-579ff0f5771d'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:25.022321 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1551de6b-e2ab-48a5-a9f5-579ff0f5771d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98c9f163ead84b2d8c847da28f3033d7', 'request_id': 'req-1551de6b-e2ab-48a5-a9f5-579ff0f5771d'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:25.086739 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a6d9b6-d3c2-46eb-8c0a-10c6613c03f7 service neutron] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:25.088275 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-d6a6d9b6-d3c2-46eb-8c0a-10c6613c03f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:25.101460 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a6d9b6-d3c2-46eb-8c0a-10c6613c03f7 service neutron] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:25.105597 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1551de6b-e2ab-48a5-a9f5-579ff0f5771d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98c9f163ead84b2d8c847da28f3033d7', 'request_id': 'req-1551de6b-e2ab-48a5-a9f5-579ff0f5771d'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:25.120227 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1551de6b-e2ab-48a5-a9f5-579ff0f5771d admin admin] The token cache is being invalidate because project 98c9f163ead84b2d8c847da28f3033d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:50:25.125234 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1551de6b-e2ab-48a5-a9f5-579ff0f5771d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:25.126275 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1551de6b-e2ab-48a5-a9f5-579ff0f5771d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:25.130683 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 950/1926] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:50:24 2022] DELETE /identity/v3/projects/98c9f163ead84b2d8c847da28f3033d7 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:25.135161 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1551de6b-e2ab-48a5-a9f5-579ff0f5771d admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:25.136825 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1551de6b-e2ab-48a5-a9f5-579ff0f5771d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:25.188729 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277aa97-86d0-4e19-a1ae-34ae43da1ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:25.190085 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277aa97-86d0-4e19-a1ae-34ae43da1ee2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:25.190552 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277aa97-86d0-4e19-a1ae-34ae43da1ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:25.191046 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277aa97-86d0-4e19-a1ae-34ae43da1ee2 admin admin] PATH_INFO: `/v3/projects/952439e0f1014185b52fddea6c65fee3` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:25.193005 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6277aa97-86d0-4e19-a1ae-34ae43da1ee2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=952439e0f1014185b52fddea6c65fee3)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:25.195649 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6277aa97-86d0-4e19-a1ae-34ae43da1ee2 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:25.263863 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1551de6b-e2ab-48a5-a9f5-579ff0f5771d admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:25.269002 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3f159-c766-4b20-a89c-408b8d800258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:25.270388 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3f159-c766-4b20-a89c-408b8d800258 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:25.270843 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3f159-c766-4b20-a89c-408b8d800258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:25.271119 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3f159-c766-4b20-a89c-408b8d800258 admin admin] PATH_INFO: `/v3/users/43dc67ea5ca74884871c43e90288372b` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:25.272622 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a3f159-c766-4b20-a89c-408b8d800258 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43dc67ea5ca74884871c43e90288372b)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:25.277113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a3f159-c766-4b20-a89c-408b8d800258 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:25.381339 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-6277aa97-86d0-4e19-a1ae-34ae43da1ee2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '952439e0f1014185b52fddea6c65fee3', 'request_id': 'req-6277aa97-86d0-4e19-a1ae-34ae43da1ee2'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:25.388575 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-6277aa97-86d0-4e19-a1ae-34ae43da1ee2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '952439e0f1014185b52fddea6c65fee3', 'request_id': 'req-6277aa97-86d0-4e19-a1ae-34ae43da1ee2'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:25.456838 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-73a3f159-c766-4b20-a89c-408b8d800258 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43dc67ea5ca74884871c43e90288372b'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:25.460016 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-6277aa97-86d0-4e19-a1ae-34ae43da1ee2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '952439e0f1014185b52fddea6c65fee3', 'request_id': 'req-6277aa97-86d0-4e19-a1ae-34ae43da1ee2'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:25.460476 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:25.471972 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-6277aa97-86d0-4e19-a1ae-34ae43da1ee2 admin admin] The token cache is being invalidate because project 952439e0f1014185b52fddea6c65fee3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:50:25.472811 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-6277aa97-86d0-4e19-a1ae-34ae43da1ee2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:25.473422 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-6277aa97-86d0-4e19-a1ae-34ae43da1ee2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:25.474806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-73a3f159-c766-4b20-a89c-408b8d800258 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43dc67ea5ca74884871c43e90288372b'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:25.474953 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 977/1927] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:50:25 2022] DELETE /identity/v3/projects/952439e0f1014185b52fddea6c65fee3 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:25.477067 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-73a3f159-c766-4b20-a89c-408b8d800258 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43dc67ea5ca74884871c43e90288372b'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:25.478836 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277aa97-86d0-4e19-a1ae-34ae43da1ee2 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:25.480254 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-6277aa97-86d0-4e19-a1ae-34ae43da1ee2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:25.484679 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-73a3f159-c766-4b20-a89c-408b8d800258 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43dc67ea5ca74884871c43e90288372b'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:25.542613 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277aa97-86d0-4e19-a1ae-34ae43da1ee2 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:25.546079 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:25.548886 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07098d7e-f2c8-46bf-ab96-a2c6c7789194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:25.549518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07098d7e-f2c8-46bf-ab96-a2c6c7789194 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:25.549799 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07098d7e-f2c8-46bf-ab96-a2c6c7789194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:25.550073 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07098d7e-f2c8-46bf-ab96-a2c6c7789194 admin admin] PATH_INFO: `/v3/users/7083bc5384ff4d7c996f95c139f23e1d` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:25.552154 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07098d7e-f2c8-46bf-ab96-a2c6c7789194 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7083bc5384ff4d7c996f95c139f23e1d)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:25.554062 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07098d7e-f2c8-46bf-ab96-a2c6c7789194 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:25.562611 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-73a3f159-c766-4b20-a89c-408b8d800258 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43dc67ea5ca74884871c43e90288372b'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:25.572771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 951/1928] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:50:25 2022] DELETE /identity/v3/users/43dc67ea5ca74884871c43e90288372b => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:25.615449 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:25.696433 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-07098d7e-f2c8-46bf-ab96-a2c6c7789194 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7083bc5384ff4d7c996f95c139f23e1d'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:25.704291 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-07098d7e-f2c8-46bf-ab96-a2c6c7789194 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7083bc5384ff4d7c996f95c139f23e1d'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:25.717105 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-07098d7e-f2c8-46bf-ab96-a2c6c7789194 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7083bc5384ff4d7c996f95c139f23e1d'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:25.717916 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-07098d7e-f2c8-46bf-ab96-a2c6c7789194 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7083bc5384ff4d7c996f95c139f23e1d'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:25.828653 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-07098d7e-f2c8-46bf-ab96-a2c6c7789194 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7083bc5384ff4d7c996f95c139f23e1d'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:25.835839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 978/1929] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:50:25 2022] DELETE /identity/v3/users/7083bc5384ff4d7c996f95c139f23e1d => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:50:26.046242 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3f159-c766-4b20-a89c-408b8d800258 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:26.046242 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-73a3f159-c766-4b20-a89c-408b8d800258 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:26.134665 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:26.135229 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3f159-c766-4b20-a89c-408b8d800258 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:26.200241 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd7e72-f508-4af8-b64d-e30c0b2e492d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:26.202266 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd7e72-f508-4af8-b64d-e30c0b2e492d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:26.202266 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd7e72-f508-4af8-b64d-e30c0b2e492d admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:26.202266 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd7e72-f508-4af8-b64d-e30c0b2e492d admin admin] PATH_INFO: `/v3/projects/65b666bb7b63475fbff28a9d208db1e1` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:26.203182 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bd7e72-f508-4af8-b64d-e30c0b2e492d admin admin] RBAC: Authorizing `identity:delete_project(project_id=65b666bb7b63475fbff28a9d208db1e1)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:26.205261 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bd7e72-f508-4af8-b64d-e30c0b2e492d admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:26.220242 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:26.357717 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d8bd7e72-f508-4af8-b64d-e30c0b2e492d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65b666bb7b63475fbff28a9d208db1e1', 'request_id': 'req-d8bd7e72-f508-4af8-b64d-e30c0b2e492d'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:26.363978 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d8bd7e72-f508-4af8-b64d-e30c0b2e492d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65b666bb7b63475fbff28a9d208db1e1', 'request_id': 'req-d8bd7e72-f508-4af8-b64d-e30c0b2e492d'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:26.450861 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07098d7e-f2c8-46bf-ab96-a2c6c7789194 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:26.452328 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-07098d7e-f2c8-46bf-ab96-a2c6c7789194 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:26.466925 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07098d7e-f2c8-46bf-ab96-a2c6c7789194 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:26.477167 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d8bd7e72-f508-4af8-b64d-e30c0b2e492d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65b666bb7b63475fbff28a9d208db1e1', 'request_id': 'req-d8bd7e72-f508-4af8-b64d-e30c0b2e492d'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:26.490485 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d8bd7e72-f508-4af8-b64d-e30c0b2e492d admin admin] The token cache is being invalidate because project 65b666bb7b63475fbff28a9d208db1e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:50:26.490784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d8bd7e72-f508-4af8-b64d-e30c0b2e492d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:26.491468 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d8bd7e72-f508-4af8-b64d-e30c0b2e492d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:26.493016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 952/1930] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:50:26 2022] DELETE /identity/v3/projects/65b666bb7b63475fbff28a9d208db1e1 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:26.496858 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd7e72-f508-4af8-b64d-e30c0b2e492d admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:26.498397 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d8bd7e72-f508-4af8-b64d-e30c0b2e492d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:26.528058 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41d8d2-8ed5-40e6-aa0c-b3ff9c4d7cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:26.528699 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41d8d2-8ed5-40e6-aa0c-b3ff9c4d7cfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:26.528981 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41d8d2-8ed5-40e6-aa0c-b3ff9c4d7cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:26.529251 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41d8d2-8ed5-40e6-aa0c-b3ff9c4d7cfa admin admin] PATH_INFO: `/v3/projects/d848e6d05516445795632b462af64f4c` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:26.530363 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f41d8d2-8ed5-40e6-aa0c-b3ff9c4d7cfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=d848e6d05516445795632b462af64f4c)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:26.532305 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f41d8d2-8ed5-40e6-aa0c-b3ff9c4d7cfa admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:26.564746 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd7e72-f508-4af8-b64d-e30c0b2e492d admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:26.571020 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b1f3ec-bcea-49d6-985c-52d25e48a52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:26.571801 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b1f3ec-bcea-49d6-985c-52d25e48a52c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:26.572078 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b1f3ec-bcea-49d6-985c-52d25e48a52c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:26.572348 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b1f3ec-bcea-49d6-985c-52d25e48a52c admin admin] PATH_INFO: `/v3/users/5de683d15f7849ab98ae0d608ca1059b` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:26.573519 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b1f3ec-bcea-49d6-985c-52d25e48a52c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5de683d15f7849ab98ae0d608ca1059b)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:26.575462 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b1f3ec-bcea-49d6-985c-52d25e48a52c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:26.745753 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-2f41d8d2-8ed5-40e6-aa0c-b3ff9c4d7cfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd848e6d05516445795632b462af64f4c', 'request_id': 'req-2f41d8d2-8ed5-40e6-aa0c-b3ff9c4d7cfa'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:26.752119 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-2f41d8d2-8ed5-40e6-aa0c-b3ff9c4d7cfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd848e6d05516445795632b462af64f4c', 'request_id': 'req-2f41d8d2-8ed5-40e6-aa0c-b3ff9c4d7cfa'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:26.854221 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-2f41d8d2-8ed5-40e6-aa0c-b3ff9c4d7cfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd848e6d05516445795632b462af64f4c', 'request_id': 'req-2f41d8d2-8ed5-40e6-aa0c-b3ff9c4d7cfa'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:26.867035 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-2f41d8d2-8ed5-40e6-aa0c-b3ff9c4d7cfa admin admin] The token cache is being invalidate because project d848e6d05516445795632b462af64f4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:50:26.867458 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-2f41d8d2-8ed5-40e6-aa0c-b3ff9c4d7cfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:26.868174 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-2f41d8d2-8ed5-40e6-aa0c-b3ff9c4d7cfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:26.869969 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 979/1931] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:50:26 2022] DELETE /identity/v3/projects/d848e6d05516445795632b462af64f4c => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:50:26.870959 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-21b1f3ec-bcea-49d6-985c-52d25e48a52c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5de683d15f7849ab98ae0d608ca1059b'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:26.874924 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434b76a-1fca-4a88-8ce4-f444d17bb081 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:26.875486 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434b76a-1fca-4a88-8ce4-f444d17bb081 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:26.875903 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434b76a-1fca-4a88-8ce4-f444d17bb081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:26.891974 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-21b1f3ec-bcea-49d6-985c-52d25e48a52c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5de683d15f7849ab98ae0d608ca1059b'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:26.892357 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-21b1f3ec-bcea-49d6-985c-52d25e48a52c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5de683d15f7849ab98ae0d608ca1059b'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:26.897068 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-a434b76a-1fca-4a88-8ce4-f444d17bb081 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:26.898271 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-a434b76a-1fca-4a88-8ce4-f444d17bb081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:26.899183 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-21b1f3ec-bcea-49d6-985c-52d25e48a52c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5de683d15f7849ab98ae0d608ca1059b'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:26.960353 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 980/1932] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:26.968363 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881645f1-e816-4307-86e2-2b7c86629615 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:26.968693 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881645f1-e816-4307-86e2-2b7c86629615 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:26.973159 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881645f1-e816-4307-86e2-2b7c86629615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:26.991048 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-21b1f3ec-bcea-49d6-985c-52d25e48a52c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5de683d15f7849ab98ae0d608ca1059b'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:26.991759 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-881645f1-e816-4307-86e2-2b7c86629615 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:26.993073 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-881645f1-e816-4307-86e2-2b7c86629615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:27.006779 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 953/1933] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:50:26 2022] DELETE /identity/v3/users/5de683d15f7849ab98ae0d608ca1059b => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:50:27.007476 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 981/1934] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:50:27.007557 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b1f3ec-bcea-49d6-985c-52d25e48a52c admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:27.082412 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b1f3ec-bcea-49d6-985c-52d25e48a52c admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:27.087958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56469a7c-d634-4a29-adcb-4224d87371f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:27.088665 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56469a7c-d634-4a29-adcb-4224d87371f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:27.089150 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56469a7c-d634-4a29-adcb-4224d87371f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:27.089584 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56469a7c-d634-4a29-adcb-4224d87371f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:27.090407 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56469a7c-d634-4a29-adcb-4224d87371f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:50:27.090718 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56469a7c-d634-4a29-adcb-4224d87371f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:27.092348 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56469a7c-d634-4a29-adcb-4224d87371f5 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:27.111367 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 954/1935] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:50:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:50:27.117197 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881645f1-e816-4307-86e2-2b7c86629615 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:27.126300 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881645f1-e816-4307-86e2-2b7c86629615 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:27.131347 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c1848-3b36-4609-a479-6701be6bca50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:27.132092 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c1848-3b36-4609-a479-6701be6bca50 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:27.132371 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c1848-3b36-4609-a479-6701be6bca50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:27.132639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c1848-3b36-4609-a479-6701be6bca50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:27.133357 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c1848-3b36-4609-a479-6701be6bca50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:27.135246 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c1848-3b36-4609-a479-6701be6bca50 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:27.231415 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 982/1936] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:50:27 2022] POST /identity/v3/projects => generated 365 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:27.234268 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56469a7c-d634-4a29-adcb-4224d87371f5 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:27.263546 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56469a7c-d634-4a29-adcb-4224d87371f5 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:27.363334 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:27.376559 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de848fb7-b534-413d-bdef-9e22a558f8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:27.377573 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de848fb7-b534-413d-bdef-9e22a558f8de admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:27.377859 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de848fb7-b534-413d-bdef-9e22a558f8de admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:27.378137 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de848fb7-b534-413d-bdef-9e22a558f8de admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:27.378918 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de848fb7-b534-413d-bdef-9e22a558f8de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:27.380982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de848fb7-b534-413d-bdef-9e22a558f8de admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:27.523467 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 955/1937] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:50:27 2022] POST /identity/v3/users => generated 283 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:27.525588 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c1848-3b36-4609-a479-6701be6bca50 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:27.526934 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de848fb7-b534-413d-bdef-9e22a558f8de admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:27.528193 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-824c1848-3b36-4609-a479-6701be6bca50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:27.539550 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de848fb7-b534-413d-bdef-9e22a558f8de admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:27.551622 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c1848-3b36-4609-a479-6701be6bca50 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:27.552937 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c3d8f-9d48-471a-ab05-b472ad817369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:27.553673 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c3d8f-9d48-471a-ab05-b472ad817369 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:27.553950 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c3d8f-9d48-471a-ab05-b472ad817369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:27.554221 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c3d8f-9d48-471a-ab05-b472ad817369 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:27.557530 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e1609-24ca-405d-8ee4-f7a662acb39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:27.557994 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6c3d8f-9d48-471a-ab05-b472ad817369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:27.558512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e1609-24ca-405d-8ee4-f7a662acb39d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:27.558800 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e1609-24ca-405d-8ee4-f7a662acb39d admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:27.559072 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e1609-24ca-405d-8ee4-f7a662acb39d admin admin] PATH_INFO: `/v3/projects/c491ea2cbe9248739252f4e9ad881125` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:27.560481 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e1609-24ca-405d-8ee4-f7a662acb39d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c491ea2cbe9248739252f4e9ad881125)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:27.562634 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e1609-24ca-405d-8ee4-f7a662acb39d admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:27.565104 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6c3d8f-9d48-471a-ab05-b472ad817369 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:27.599044 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:27.607670 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 956/1938] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 12:50:27.611683 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c3d8f-9d48-471a-ab05-b472ad817369 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:27.628272 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c3d8f-9d48-471a-ab05-b472ad817369 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:27.634296 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6458d5b5-501c-45e9-a24e-8ce5badda131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:27.635156 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458d5b5-501c-45e9-a24e-8ce5badda131 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:27.635549 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458d5b5-501c-45e9-a24e-8ce5badda131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:27.635890 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458d5b5-501c-45e9-a24e-8ce5badda131 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:27.636608 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6458d5b5-501c-45e9-a24e-8ce5badda131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:27.638741 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6458d5b5-501c-45e9-a24e-8ce5badda131 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:27.653713 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 957/1939] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:50:27.658078 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6458d5b5-501c-45e9-a24e-8ce5badda131 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:27.673995 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6458d5b5-501c-45e9-a24e-8ce5badda131 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:27.681491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b54b84-a718-4df1-a7d9-82e5526a7dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:27.682349 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b54b84-a718-4df1-a7d9-82e5526a7dbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:27.683050 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b54b84-a718-4df1-a7d9-82e5526a7dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:27.688449 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b54b84-a718-4df1-a7d9-82e5526a7dbe admin admin] PATH_INFO: `/v3/projects/fdbd476d74bc4d1aad15ef922d9cbd54/users/c45303e113f34b3e8d710057074eaafe/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:27.704796 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b54b84-a718-4df1-a7d9-82e5526a7dbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdbd476d74bc4d1aad15ef922d9cbd54, user_id=c45303e113f34b3e8d710057074eaafe, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:27.710511 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b54b84-a718-4df1-a7d9-82e5526a7dbe admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:27.740887 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-0d2e1609-24ca-405d-8ee4-f7a662acb39d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c491ea2cbe9248739252f4e9ad881125', 'request_id': 'req-0d2e1609-24ca-405d-8ee4-f7a662acb39d'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:27.754707 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-0d2e1609-24ca-405d-8ee4-f7a662acb39d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c491ea2cbe9248739252f4e9ad881125', 'request_id': 'req-0d2e1609-24ca-405d-8ee4-f7a662acb39d'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:27.793206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 958/1940] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:50:27 2022] PUT /identity/v3/projects/fdbd476d74bc4d1aad15ef922d9cbd54/users/c45303e113f34b3e8d710057074eaafe/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:50:27.793206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b54b84-a718-4df1-a7d9-82e5526a7dbe admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:27.803846 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-69b54b84-a718-4df1-a7d9-82e5526a7dbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:27.866036 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-0d2e1609-24ca-405d-8ee4-f7a662acb39d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c491ea2cbe9248739252f4e9ad881125', 'request_id': 'req-0d2e1609-24ca-405d-8ee4-f7a662acb39d'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:27.868032 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b54b84-a718-4df1-a7d9-82e5526a7dbe admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:27.872452 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adf9ac2-c2cf-49c2-9cce-a7f6a8614aae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:27.873066 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf9ac2-c2cf-49c2-9cce-a7f6a8614aae service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:27.873355 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf9ac2-c2cf-49c2-9cce-a7f6a8614aae service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:27.873638 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf9ac2-c2cf-49c2-9cce-a7f6a8614aae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:27.875398 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4adf9ac2-c2cf-49c2-9cce-a7f6a8614aae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:27.882290 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-0d2e1609-24ca-405d-8ee4-f7a662acb39d admin admin] The token cache is being invalidate because project c491ea2cbe9248739252f4e9ad881125 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:50:27.882801 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-0d2e1609-24ca-405d-8ee4-f7a662acb39d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:27.884387 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-0d2e1609-24ca-405d-8ee4-f7a662acb39d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:27.886669 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 983/1941] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:50:27 2022] DELETE /identity/v3/projects/c491ea2cbe9248739252f4e9ad881125 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:27.888367 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b5529-1c4d-45d4-8d47-bbb8ed0eff6f None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:27.888658 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b5529-1c4d-45d4-8d47-bbb8ed0eff6f None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:27.888939 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b5529-1c4d-45d4-8d47-bbb8ed0eff6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:27.925228 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-a25b5529-1c4d-45d4-8d47-bbb8ed0eff6f None None] MFA Rules not processed for user `c45303e113f34b3e8d710057074eaafe`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:27.926862 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-a25b5529-1c4d-45d4-8d47-bbb8ed0eff6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:27.954826 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adf9ac2-c2cf-49c2-9cce-a7f6a8614aae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:27.956413 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adf9ac2-c2cf-49c2-9cce-a7f6a8614aae service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:27.970108 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 959/1942] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:50:27 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:50:27.971463 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536d15d-9bba-40b7-8b3e-bd82188fdcbd None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:27.971776 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536d15d-9bba-40b7-8b3e-bd82188fdcbd None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:27.972530 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536d15d-9bba-40b7-8b3e-bd82188fdcbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:28.013191 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-9536d15d-9bba-40b7-8b3e-bd82188fdcbd None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:28.013191 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-9536d15d-9bba-40b7-8b3e-bd82188fdcbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:28.035386 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 984/1943] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:28.035386 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b5529-1c4d-45d4-8d47-bbb8ed0eff6f None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:28.038348 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-a25b5529-1c4d-45d4-8d47-bbb8ed0eff6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:28.125202 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 960/1944] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:28.134099 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192d1ab-e17d-4515-baf5-d6673c81bb7e None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:28.134406 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192d1ab-e17d-4515-baf5-d6673c81bb7e None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:28.134682 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192d1ab-e17d-4515-baf5-d6673c81bb7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:28.138569 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b5529-1c4d-45d4-8d47-bbb8ed0eff6f None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:28.143584 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9050d3-6281-4cae-9b84-d1b27da38e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:28.144979 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9050d3-6281-4cae-9b84-d1b27da38e43 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:28.145552 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9050d3-6281-4cae-9b84-d1b27da38e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:28.147229 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9050d3-6281-4cae-9b84-d1b27da38e43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:28.147956 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9050d3-6281-4cae-9b84-d1b27da38e43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:28.150210 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9050d3-6281-4cae-9b84-d1b27da38e43 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:28.158188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-0192d1ab-e17d-4515-baf5-d6673c81bb7e None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:28.159608 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-0192d1ab-e17d-4515-baf5-d6673c81bb7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:28.170091 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 961/1945] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:28.174443 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0192d1ab-e17d-4515-baf5-d6673c81bb7e None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:28.185216 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0192d1ab-e17d-4515-baf5-d6673c81bb7e None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:28.190789 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8c4a9-0e68-40c8-8037-1c2b515c9eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:28.191499 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8c4a9-0e68-40c8-8037-1c2b515c9eac admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:28.191772 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8c4a9-0e68-40c8-8037-1c2b515c9eac admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:28.192039 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8c4a9-0e68-40c8-8037-1c2b515c9eac admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:28.193912 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e8c4a9-0e68-40c8-8037-1c2b515c9eac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:50:28.193912 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e8c4a9-0e68-40c8-8037-1c2b515c9eac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:28.195104 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e8c4a9-0e68-40c8-8037-1c2b515c9eac admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:28.207928 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 962/1946] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:50:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:50:28.212488 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8c4a9-0e68-40c8-8037-1c2b515c9eac admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:28.223606 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8c4a9-0e68-40c8-8037-1c2b515c9eac admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:28.228297 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e28088-872d-4f48-bfb6-3c814daba3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:28.228930 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e28088-872d-4f48-bfb6-3c814daba3e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:28.229201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e28088-872d-4f48-bfb6-3c814daba3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:28.229476 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e28088-872d-4f48-bfb6-3c814daba3e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:28.230103 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e28088-872d-4f48-bfb6-3c814daba3e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:28.232121 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e28088-872d-4f48-bfb6-3c814daba3e5 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:28.244735 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 985/1947] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:50:28 2022] POST /identity/v3/projects => generated 365 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:28.248739 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9050d3-6281-4cae-9b84-d1b27da38e43 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:28.350474 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9050d3-6281-4cae-9b84-d1b27da38e43 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:28.359244 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:28.368297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93a339-6a55-44a8-bcf8-b0ba87f25309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:28.369256 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93a339-6a55-44a8-bcf8-b0ba87f25309 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:28.369705 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93a339-6a55-44a8-bcf8-b0ba87f25309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:28.370173 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93a339-6a55-44a8-bcf8-b0ba87f25309 admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:28.371651 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb93a339-6a55-44a8-bcf8-b0ba87f25309 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:28.372434 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 963/1948] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:50:28 2022] POST /identity/v3/projects => generated 385 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:50:28.374397 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb93a339-6a55-44a8-bcf8-b0ba87f25309 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:28.375810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e28088-872d-4f48-bfb6-3c814daba3e5 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:28.444719 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e28088-872d-4f48-bfb6-3c814daba3e5 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:28.449705 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b8b70-e644-483e-acad-8b3398c2bf68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:28.450360 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b8b70-e644-483e-acad-8b3398c2bf68 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:28.450650 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b8b70-e644-483e-acad-8b3398c2bf68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:28.450916 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b8b70-e644-483e-acad-8b3398c2bf68 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:28.451575 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7b8b70-e644-483e-acad-8b3398c2bf68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:28.453494 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7b8b70-e644-483e-acad-8b3398c2bf68 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:28.495669 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 986/1949] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:50:28 2022] POST /identity/v3/users => generated 289 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:50:28.499618 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93a339-6a55-44a8-bcf8-b0ba87f25309 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:28.510008 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93a339-6a55-44a8-bcf8-b0ba87f25309 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:28.514797 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae75d8-4d13-4df9-8999-961091f95603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:28.515446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae75d8-4d13-4df9-8999-961091f95603 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:28.515731 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae75d8-4d13-4df9-8999-961091f95603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:28.516006 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae75d8-4d13-4df9-8999-961091f95603 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:28.516651 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae75d8-4d13-4df9-8999-961091f95603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:28.518281 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae75d8-4d13-4df9-8999-961091f95603 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:28.533380 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 987/1950] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 12:50:28.534220 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae75d8-4d13-4df9-8999-961091f95603 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:28.545826 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae75d8-4d13-4df9-8999-961091f95603 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:28.551924 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc67a1a-6e9c-4f7d-8a17-5b34270d6ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:28.552586 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc67a1a-6e9c-4f7d-8a17-5b34270d6ec2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:28.552871 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc67a1a-6e9c-4f7d-8a17-5b34270d6ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:28.553149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc67a1a-6e9c-4f7d-8a17-5b34270d6ec2 admin admin] PATH_INFO: `/v3/projects/3fe8f88ee77e472ba1d75108c5c12aa9/users/4b1fcba0717f4730a92267a91f9ee4c0/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:28.569317 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc67a1a-6e9c-4f7d-8a17-5b34270d6ec2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fe8f88ee77e472ba1d75108c5c12aa9, user_id=4b1fcba0717f4730a92267a91f9ee4c0, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:28.569511 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 964/1951] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:50:28 2022] POST /identity/v3/users => generated 293 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:28.574559 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc67a1a-6e9c-4f7d-8a17-5b34270d6ec2 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:28.574680 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b8b70-e644-483e-acad-8b3398c2bf68 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:28.585015 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b8b70-e644-483e-acad-8b3398c2bf68 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:28.590442 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156bada1-c601-40e1-8c2b-195b80e1bea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:28.591353 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156bada1-c601-40e1-8c2b-195b80e1bea0 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:28.591471 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156bada1-c601-40e1-8c2b-195b80e1bea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:28.591674 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156bada1-c601-40e1-8c2b-195b80e1bea0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:28.594096 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156bada1-c601-40e1-8c2b-195b80e1bea0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:28.594199 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156bada1-c601-40e1-8c2b-195b80e1bea0 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:28.606161 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 965/1952] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 12:50:28.610018 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156bada1-c601-40e1-8c2b-195b80e1bea0 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:28.620639 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156bada1-c601-40e1-8c2b-195b80e1bea0 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:28.625300 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b885ae77-6b3b-4ebe-b415-48d813e30ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:28.625904 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885ae77-6b3b-4ebe-b415-48d813e30ee8 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:28.626188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885ae77-6b3b-4ebe-b415-48d813e30ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:28.626468 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885ae77-6b3b-4ebe-b415-48d813e30ee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:28.627106 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b885ae77-6b3b-4ebe-b415-48d813e30ee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:28.629640 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b885ae77-6b3b-4ebe-b415-48d813e30ee8 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:28.642246 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 966/1953] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 12:50:28.647047 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b885ae77-6b3b-4ebe-b415-48d813e30ee8 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:28.658601 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b885ae77-6b3b-4ebe-b415-48d813e30ee8 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:28.664679 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31d483-a4c9-498d-87d4-13de27bdd64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:28.665349 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31d483-a4c9-498d-87d4-13de27bdd64b admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:28.665646 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31d483-a4c9-498d-87d4-13de27bdd64b admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:28.665906 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31d483-a4c9-498d-87d4-13de27bdd64b admin admin] PATH_INFO: `/v3/projects/d14368f1a3b44ffcac4d517610e3552b/users/51f862cff85949af9feff21a8f8c9aed/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:28.685926 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c31d483-a4c9-498d-87d4-13de27bdd64b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d14368f1a3b44ffcac4d517610e3552b, user_id=51f862cff85949af9feff21a8f8c9aed, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:28.688680 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c31d483-a4c9-498d-87d4-13de27bdd64b admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:28.695209 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 988/1954] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:50:28 2022] PUT /identity/v3/projects/3fe8f88ee77e472ba1d75108c5c12aa9/users/4b1fcba0717f4730a92267a91f9ee4c0/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:50:28.705151 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc67a1a-6e9c-4f7d-8a17-5b34270d6ec2 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:28.774860 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:28.787911 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc67a1a-6e9c-4f7d-8a17-5b34270d6ec2 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:28.798459 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 967/1955] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:50:28 2022] PUT /identity/v3/projects/d14368f1a3b44ffcac4d517610e3552b/users/51f862cff85949af9feff21a8f8c9aed/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:50:28.799980 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e43846-3637-41b8-9663-457a3ff6be27 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:28.800267 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e43846-3637-41b8-9663-457a3ff6be27 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:28.801061 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e43846-3637-41b8-9663-457a3ff6be27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:28.841005 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-39e43846-3637-41b8-9663-457a3ff6be27 None None] MFA Rules not processed for user `51f862cff85949af9feff21a8f8c9aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:28.842247 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-39e43846-3637-41b8-9663-457a3ff6be27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:28.870126 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20ae623-a658-44de-be7d-e22b659faf69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:28.872136 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ae623-a658-44de-be7d-e22b659faf69 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:28.872439 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ae623-a658-44de-be7d-e22b659faf69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:28.872722 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ae623-a658-44de-be7d-e22b659faf69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:28.873382 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20ae623-a658-44de-be7d-e22b659faf69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:28.880383 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20ae623-a658-44de-be7d-e22b659faf69 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:28.901352 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 989/1956] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 12:50:28.905419 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20ae623-a658-44de-be7d-e22b659faf69 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:28.925996 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20ae623-a658-44de-be7d-e22b659faf69 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:28.928898 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca6fbec-db5b-4acd-a603-161ddb709e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:28.929673 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6fbec-db5b-4acd-a603-161ddb709e3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:28.930022 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6fbec-db5b-4acd-a603-161ddb709e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:28.930356 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6fbec-db5b-4acd-a603-161ddb709e3a admin admin] PATH_INFO: `/v3/projects/3fe8f88ee77e472ba1d75108c5c12aa9/users/4b1fcba0717f4730a92267a91f9ee4c0/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:28.933968 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca6fbec-db5b-4acd-a603-161ddb709e3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fe8f88ee77e472ba1d75108c5c12aa9, user_id=4b1fcba0717f4730a92267a91f9ee4c0, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:28.938139 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca6fbec-db5b-4acd-a603-161ddb709e3a admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:28.951435 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 990/1957] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:50:28 2022] PUT /identity/v3/projects/3fe8f88ee77e472ba1d75108c5c12aa9/users/4b1fcba0717f4730a92267a91f9ee4c0/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:28.962450 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb9ee6-e29a-40f4-b541-26af51fb33ab None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:28.962731 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb9ee6-e29a-40f4-b541-26af51fb33ab None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:28.963002 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb9ee6-e29a-40f4-b541-26af51fb33ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:28.968776 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 968/1958] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:28.974593 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e43846-3637-41b8-9663-457a3ff6be27 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:28.991995 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e43846-3637-41b8-9663-457a3ff6be27 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:29.004310 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-a8bb9ee6-e29a-40f4-b541-26af51fb33ab None None] MFA Rules not processed for user `4b1fcba0717f4730a92267a91f9ee4c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:29.005881 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-a8bb9ee6-e29a-40f4-b541-26af51fb33ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:29.083536 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 991/1959] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:28 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:29.091879 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f846a6b5-0a07-4cf1-928c-31a2ca81f559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:29.092580 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f846a6b5-0a07-4cf1-928c-31a2ca81f559 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:29.092861 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f846a6b5-0a07-4cf1-928c-31a2ca81f559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:29.093136 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f846a6b5-0a07-4cf1-928c-31a2ca81f559 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:29.093774 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f846a6b5-0a07-4cf1-928c-31a2ca81f559 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:29.096040 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f846a6b5-0a07-4cf1-928c-31a2ca81f559 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:29.181596 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 969/1960] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:50:28 2022] POST /identity/v3/projects => generated 387 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:50:29.185669 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bb9ee6-e29a-40f4-b541-26af51fb33ab None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:29.197194 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bb9ee6-e29a-40f4-b541-26af51fb33ab None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:29.268381 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a5e1f-4af1-439b-a90d-2957c849046e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:29.269009 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a5e1f-4af1-439b-a90d-2957c849046e admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:29.269566 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a5e1f-4af1-439b-a90d-2957c849046e admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:29.270060 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a5e1f-4af1-439b-a90d-2957c849046e admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:29.270714 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7a5e1f-4af1-439b-a90d-2957c849046e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:29.272749 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f846a6b5-0a07-4cf1-928c-31a2ca81f559 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:29.272893 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7a5e1f-4af1-439b-a90d-2957c849046e admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:29.274641 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-f846a6b5-0a07-4cf1-928c-31a2ca81f559 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:29.338751 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f846a6b5-0a07-4cf1-928c-31a2ca81f559 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:29.340822 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d40a7fc-f275-451f-8806-1b2d124bbc13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:29.341452 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40a7fc-f275-451f-8806-1b2d124bbc13 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:29.341732 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40a7fc-f275-451f-8806-1b2d124bbc13 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:29.342010 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40a7fc-f275-451f-8806-1b2d124bbc13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:29.412536 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:29.414688 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 992/1961] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:50:29 2022] POST /identity/v3/users => generated 300 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:29.415671 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a5e1f-4af1-439b-a90d-2957c849046e admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:29.416363 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d40a7fc-f275-451f-8806-1b2d124bbc13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:29.417870 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d40a7fc-f275-451f-8806-1b2d124bbc13 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:29.426843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a5e1f-4af1-439b-a90d-2957c849046e admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:29.429602 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 970/1962] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:50:29 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:50:29.433903 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1424931a-9aa4-4c82-8ead-2d04caa1f9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:29.434698 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1424931a-9aa4-4c82-8ead-2d04caa1f9fb admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:29.434983 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1424931a-9aa4-4c82-8ead-2d04caa1f9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:29.435254 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1424931a-9aa4-4c82-8ead-2d04caa1f9fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:29.436380 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1424931a-9aa4-4c82-8ead-2d04caa1f9fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:29.437711 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1424931a-9aa4-4c82-8ead-2d04caa1f9fb admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:29.449449 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 993/1963] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:50:29.455645 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d40a7fc-f275-451f-8806-1b2d124bbc13 service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:29.468610 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d40a7fc-f275-451f-8806-1b2d124bbc13 service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:29.474731 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2711173-5f44-4c8d-b258-cb24daa7a581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:29.475521 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2711173-5f44-4c8d-b258-cb24daa7a581 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:29.475778 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2711173-5f44-4c8d-b258-cb24daa7a581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:29.476055 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2711173-5f44-4c8d-b258-cb24daa7a581 admin admin] PATH_INFO: `/v3/projects/e1a3553ed45145ff8062098e6c1b5606/users/567b9864b12c44ad8e8728fec640fcfa/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:29.492753 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2711173-5f44-4c8d-b258-cb24daa7a581 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1a3553ed45145ff8062098e6c1b5606, user_id=567b9864b12c44ad8e8728fec640fcfa, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:29.495695 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2711173-5f44-4c8d-b258-cb24daa7a581 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:29.706145 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:29.717385 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 971/1964] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:50:29 2022] PUT /identity/v3/projects/e1a3553ed45145ff8062098e6c1b5606/users/567b9864b12c44ad8e8728fec640fcfa/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:29.718606 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1424931a-9aa4-4c82-8ead-2d04caa1f9fb admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:29.733017 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1424931a-9aa4-4c82-8ead-2d04caa1f9fb admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:29.805540 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2711173-5f44-4c8d-b258-cb24daa7a581 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:29.838947 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba07f6cd-5245-49c5-827a-4922ec999063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:29.839781 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07f6cd-5245-49c5-827a-4922ec999063 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:29.840189 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07f6cd-5245-49c5-827a-4922ec999063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:29.840641 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07f6cd-5245-49c5-827a-4922ec999063 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:29.841696 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba07f6cd-5245-49c5-827a-4922ec999063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:29.843631 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba07f6cd-5245-49c5-827a-4922ec999063 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:29.854346 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 994/1965] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:50:29.858390 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba07f6cd-5245-49c5-827a-4922ec999063 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:29.871089 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba07f6cd-5245-49c5-827a-4922ec999063 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:29.873827 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2711173-5f44-4c8d-b258-cb24daa7a581 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:29.876966 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d49c4d-37c5-43ce-9b5b-51383ebd2d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:29.877720 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e4b2b-44c7-4c91-bd2f-4fa535903f63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:29.877869 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d49c4d-37c5-43ce-9b5b-51383ebd2d98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:29.878120 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d49c4d-37c5-43ce-9b5b-51383ebd2d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:29.878336 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e4b2b-44c7-4c91-bd2f-4fa535903f63 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:29.878585 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d49c4d-37c5-43ce-9b5b-51383ebd2d98 admin admin] PATH_INFO: `/v3/projects/e1a3553ed45145ff8062098e6c1b5606/users/567b9864b12c44ad8e8728fec640fcfa/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:29.878690 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e4b2b-44c7-4c91-bd2f-4fa535903f63 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:29.879082 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e4b2b-44c7-4c91-bd2f-4fa535903f63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:29.880809 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d49c4d-37c5-43ce-9b5b-51383ebd2d98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1a3553ed45145ff8062098e6c1b5606, user_id=567b9864b12c44ad8e8728fec640fcfa, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:29.883124 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d49c4d-37c5-43ce-9b5b-51383ebd2d98 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:29.911355 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 995/1966] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:50:29 2022] PUT /identity/v3/projects/e1a3553ed45145ff8062098e6c1b5606/users/567b9864b12c44ad8e8728fec640fcfa/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:50:29.922439 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41479e4c-9f22-4f5b-810d-df3b3ed68086 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:29.922774 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41479e4c-9f22-4f5b-810d-df3b3ed68086 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:29.922985 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41479e4c-9f22-4f5b-810d-df3b3ed68086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:29.955381 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117e4b2b-44c7-4c91-bd2f-4fa535903f63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:29.957394 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117e4b2b-44c7-4c91-bd2f-4fa535903f63 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:29.963275 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-41479e4c-9f22-4f5b-810d-df3b3ed68086 None None] MFA Rules not processed for user `567b9864b12c44ad8e8728fec640fcfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:29.970604 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-41479e4c-9f22-4f5b-810d-df3b3ed68086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:29.970748 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 972/1967] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:50:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:50:30.037817 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 996/1968] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:29 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:50:30.191844 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:30.249908 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e4b2b-44c7-4c91-bd2f-4fa535903f63 service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:30.252227 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-117e4b2b-44c7-4c91-bd2f-4fa535903f63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:30.265020 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e4b2b-44c7-4c91-bd2f-4fa535903f63 service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:30.328861 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129290f5-d54f-4126-95f3-f6e236737e35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:30.329586 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129290f5-d54f-4126-95f3-f6e236737e35 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:30.329929 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129290f5-d54f-4126-95f3-f6e236737e35 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:30.330269 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129290f5-d54f-4126-95f3-f6e236737e35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:30.341134 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129290f5-d54f-4126-95f3-f6e236737e35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:30.342572 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129290f5-d54f-4126-95f3-f6e236737e35 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:30.356996 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 973/1969] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:50:30 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:50:30.454357 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:30.493817 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41479e4c-9f22-4f5b-810d-df3b3ed68086 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:30.513046 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41479e4c-9f22-4f5b-810d-df3b3ed68086 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:30.513046 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c461e6-6b19-49f2-9b44-64f9c6dae45d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:30.513046 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c461e6-6b19-49f2-9b44-64f9c6dae45d service cinder] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:30.513046 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c461e6-6b19-49f2-9b44-64f9c6dae45d service cinder] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:30.513413 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c461e6-6b19-49f2-9b44-64f9c6dae45d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:30.514778 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-28c461e6-6b19-49f2-9b44-64f9c6dae45d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:30.584975 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c461e6-6b19-49f2-9b44-64f9c6dae45d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:30.586520 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c461e6-6b19-49f2-9b44-64f9c6dae45d service cinder] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:30.598263 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 997/1970] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:50:30 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:50:31.054241 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:31.271639 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:31.704177 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129290f5-d54f-4126-95f3-f6e236737e35 service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:31.771308 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129290f5-d54f-4126-95f3-f6e236737e35 service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:31.776337 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3555a8f-e3cd-4dce-8040-149d98e9bf3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:31.777026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3555a8f-e3cd-4dce-8040-149d98e9bf3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:31.777299 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3555a8f-e3cd-4dce-8040-149d98e9bf3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:31.777566 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3555a8f-e3cd-4dce-8040-149d98e9bf3e admin admin] PATH_INFO: `/v3/users/51f862cff85949af9feff21a8f8c9aed` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:31.778672 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3555a8f-e3cd-4dce-8040-149d98e9bf3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=51f862cff85949af9feff21a8f8c9aed)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:31.780660 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3555a8f-e3cd-4dce-8040-149d98e9bf3e admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:31.905293 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:31.963621 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d3555a8f-e3cd-4dce-8040-149d98e9bf3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51f862cff85949af9feff21a8f8c9aed'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:31.977026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d3555a8f-e3cd-4dce-8040-149d98e9bf3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51f862cff85949af9feff21a8f8c9aed'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:31.977793 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d3555a8f-e3cd-4dce-8040-149d98e9bf3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51f862cff85949af9feff21a8f8c9aed'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:31.991949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d3555a8f-e3cd-4dce-8040-149d98e9bf3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51f862cff85949af9feff21a8f8c9aed'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:32.093827 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d3555a8f-e3cd-4dce-8040-149d98e9bf3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51f862cff85949af9feff21a8f8c9aed'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:32.100454 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 974/1971] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:50:31 2022] DELETE /identity/v3/users/51f862cff85949af9feff21a8f8c9aed => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:50:32.104973 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154636b-f3fa-406b-b864-e74807df3a5a None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:32.105454 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154636b-f3fa-406b-b864-e74807df3a5a None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:32.106327 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154636b-f3fa-406b-b864-e74807df3a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:32.125531 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-e154636b-f3fa-406b-b864-e74807df3a5a None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:32.126735 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-e154636b-f3fa-406b-b864-e74807df3a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:32.188174 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 998/1972] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:50:32.198735 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3555a8f-e3cd-4dce-8040-149d98e9bf3e admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:32.200273 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d3555a8f-e3cd-4dce-8040-149d98e9bf3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:32.279884 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3555a8f-e3cd-4dce-8040-149d98e9bf3e admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:32.285413 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6b4b5c-8937-490d-9372-68eaf1f6b2b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:32.286073 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b4b5c-8937-490d-9372-68eaf1f6b2b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:32.286894 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b4b5c-8937-490d-9372-68eaf1f6b2b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:32.287610 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b4b5c-8937-490d-9372-68eaf1f6b2b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:32.298111 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6b4b5c-8937-490d-9372-68eaf1f6b2b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:32.301101 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6b4b5c-8937-490d-9372-68eaf1f6b2b9 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:32.312288 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 975/1973] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:50:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:50:32.827561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154636b-f3fa-406b-b864-e74807df3a5a None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:32.829589 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-e154636b-f3fa-406b-b864-e74807df3a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:32.841558 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154636b-f3fa-406b-b864-e74807df3a5a None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:32.846675 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c0da7d-6278-4cd9-82a3-a2ec827ec44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:32.849886 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c0da7d-6278-4cd9-82a3-a2ec827ec44c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:32.849886 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c0da7d-6278-4cd9-82a3-a2ec827ec44c admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:32.849886 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c0da7d-6278-4cd9-82a3-a2ec827ec44c admin admin] PATH_INFO: `/v3/projects/d14368f1a3b44ffcac4d517610e3552b` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:32.849886 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c0da7d-6278-4cd9-82a3-a2ec827ec44c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d14368f1a3b44ffcac4d517610e3552b)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:32.850976 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c0da7d-6278-4cd9-82a3-a2ec827ec44c admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:32.964870 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-77c0da7d-6278-4cd9-82a3-a2ec827ec44c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd14368f1a3b44ffcac4d517610e3552b', 'request_id': 'req-77c0da7d-6278-4cd9-82a3-a2ec827ec44c'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:32.971537 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-77c0da7d-6278-4cd9-82a3-a2ec827ec44c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd14368f1a3b44ffcac4d517610e3552b', 'request_id': 'req-77c0da7d-6278-4cd9-82a3-a2ec827ec44c'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:33.059844 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-77c0da7d-6278-4cd9-82a3-a2ec827ec44c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd14368f1a3b44ffcac4d517610e3552b', 'request_id': 'req-77c0da7d-6278-4cd9-82a3-a2ec827ec44c'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:33.071124 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-77c0da7d-6278-4cd9-82a3-a2ec827ec44c admin admin] The token cache is being invalidate because project d14368f1a3b44ffcac4d517610e3552b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:50:33.071461 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-77c0da7d-6278-4cd9-82a3-a2ec827ec44c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:33.072099 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-77c0da7d-6278-4cd9-82a3-a2ec827ec44c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:33.073584 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 999/1974] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:50:32 2022] DELETE /identity/v3/projects/d14368f1a3b44ffcac4d517610e3552b => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:33.077665 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6b4b5c-8937-490d-9372-68eaf1f6b2b9 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:33.083920 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:33.087838 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6b4b5c-8937-490d-9372-68eaf1f6b2b9 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:33.144215 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa17186-a536-4846-8f16-f94d5ca6a6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:33.144936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa17186-a536-4846-8f16-f94d5ca6a6d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:33.145212 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa17186-a536-4846-8f16-f94d5ca6a6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:33.145489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa17186-a536-4846-8f16-f94d5ca6a6d3 admin admin] PATH_INFO: `/v3/users/567b9864b12c44ad8e8728fec640fcfa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:33.146598 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa17186-a536-4846-8f16-f94d5ca6a6d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=567b9864b12c44ad8e8728fec640fcfa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:33.148544 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa17186-a536-4846-8f16-f94d5ca6a6d3 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:33.327842 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-bfa17186-a536-4846-8f16-f94d5ca6a6d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '567b9864b12c44ad8e8728fec640fcfa'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:33.341673 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-bfa17186-a536-4846-8f16-f94d5ca6a6d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '567b9864b12c44ad8e8728fec640fcfa'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:33.342518 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-bfa17186-a536-4846-8f16-f94d5ca6a6d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '567b9864b12c44ad8e8728fec640fcfa'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:33.350383 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-bfa17186-a536-4846-8f16-f94d5ca6a6d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '567b9864b12c44ad8e8728fec640fcfa'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:33.437048 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-bfa17186-a536-4846-8f16-f94d5ca6a6d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '567b9864b12c44ad8e8728fec640fcfa'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:33.444735 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 976/1975] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:50:33 2022] DELETE /identity/v3/users/567b9864b12c44ad8e8728fec640fcfa => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:34.052924 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c0da7d-6278-4cd9-82a3-a2ec827ec44c admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:34.054391 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-77c0da7d-6278-4cd9-82a3-a2ec827ec44c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:34.075343 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c0da7d-6278-4cd9-82a3-a2ec827ec44c admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:34.117650 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:34.146243 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df07d75-d14b-4d9f-9f5e-36b76acab5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:34.151199 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df07d75-d14b-4d9f-9f5e-36b76acab5cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:34.151199 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df07d75-d14b-4d9f-9f5e-36b76acab5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:34.151199 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df07d75-d14b-4d9f-9f5e-36b76acab5cf admin admin] PATH_INFO: `/v3/projects/e1a3553ed45145ff8062098e6c1b5606` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:34.151199 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df07d75-d14b-4d9f-9f5e-36b76acab5cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1a3553ed45145ff8062098e6c1b5606)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:34.151858 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df07d75-d14b-4d9f-9f5e-36b76acab5cf admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:34.283127 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-3df07d75-d14b-4d9f-9f5e-36b76acab5cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1a3553ed45145ff8062098e6c1b5606', 'request_id': 'req-3df07d75-d14b-4d9f-9f5e-36b76acab5cf'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:34.289455 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-3df07d75-d14b-4d9f-9f5e-36b76acab5cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1a3553ed45145ff8062098e6c1b5606', 'request_id': 'req-3df07d75-d14b-4d9f-9f5e-36b76acab5cf'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:34.351094 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-3df07d75-d14b-4d9f-9f5e-36b76acab5cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1a3553ed45145ff8062098e6c1b5606', 'request_id': 'req-3df07d75-d14b-4d9f-9f5e-36b76acab5cf'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:34.361822 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-3df07d75-d14b-4d9f-9f5e-36b76acab5cf admin admin] The token cache is being invalidate because project e1a3553ed45145ff8062098e6c1b5606 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:50:34.362109 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-3df07d75-d14b-4d9f-9f5e-36b76acab5cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:34.363394 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-3df07d75-d14b-4d9f-9f5e-36b76acab5cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:34.364887 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1000/1976] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:50:34 2022] DELETE /identity/v3/projects/e1a3553ed45145ff8062098e6c1b5606 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:34.370048 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ab77f-665f-4412-8e54-e487f8cb7e07 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:34.370603 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ab77f-665f-4412-8e54-e487f8cb7e07 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:34.370603 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ab77f-665f-4412-8e54-e487f8cb7e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:34.396155 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-bc7ab77f-665f-4412-8e54-e487f8cb7e07 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:34.397365 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-bc7ab77f-665f-4412-8e54-e487f8cb7e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:34.458910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 977/1977] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:34.466729 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e712c-39a5-4604-a5c1-7cf9c7e79bf4 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:34.467001 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e712c-39a5-4604-a5c1-7cf9c7e79bf4 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:34.467278 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e712c-39a5-4604-a5c1-7cf9c7e79bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:34.489714 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-219e712c-39a5-4604-a5c1-7cf9c7e79bf4 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:34.490929 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-219e712c-39a5-4604-a5c1-7cf9c7e79bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:34.499482 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1001/1978] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:34.504659 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ab77f-665f-4412-8e54-e487f8cb7e07 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:34.515214 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ab77f-665f-4412-8e54-e487f8cb7e07 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:34.521532 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b7c12-7bab-416c-a3a0-f3314703da97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:34.522060 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b7c12-7bab-416c-a3a0-f3314703da97 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:34.522339 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b7c12-7bab-416c-a3a0-f3314703da97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:34.522889 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b7c12-7bab-416c-a3a0-f3314703da97 admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:34.523820 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9b7c12-7bab-416c-a3a0-f3314703da97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:50:34.524144 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9b7c12-7bab-416c-a3a0-f3314703da97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:34.525884 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9b7c12-7bab-416c-a3a0-f3314703da97 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:34.542396 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 978/1979] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:50:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:50:34.547643 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219e712c-39a5-4604-a5c1-7cf9c7e79bf4 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:34.557514 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219e712c-39a5-4604-a5c1-7cf9c7e79bf4 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:34.575010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d80312-b4dd-44f4-8cdf-793f4f095f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:34.576241 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d80312-b4dd-44f4-8cdf-793f4f095f7a admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:34.577327 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d80312-b4dd-44f4-8cdf-793f4f095f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:34.578122 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d80312-b4dd-44f4-8cdf-793f4f095f7a admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:34.579079 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d80312-b4dd-44f4-8cdf-793f4f095f7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:34.581524 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d80312-b4dd-44f4-8cdf-793f4f095f7a admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:34.680315 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1002/1980] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:50:34 2022] POST /identity/v3/projects => generated 373 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:34.681333 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b7c12-7bab-416c-a3a0-f3314703da97 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:34.695488 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b7c12-7bab-416c-a3a0-f3314703da97 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:34.711459 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:34.811355 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e2158-dfa9-457d-b792-0b0b5f8dc83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:34.812081 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e2158-dfa9-457d-b792-0b0b5f8dc83c admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:34.812377 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e2158-dfa9-457d-b792-0b0b5f8dc83c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:34.812828 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e2158-dfa9-457d-b792-0b0b5f8dc83c admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:34.813479 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2e2158-dfa9-457d-b792-0b0b5f8dc83c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:34.818024 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2e2158-dfa9-457d-b792-0b0b5f8dc83c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:34.902987 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 979/1981] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:50:34 2022] POST /identity/v3/users => generated 287 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:34.907045 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d80312-b4dd-44f4-8cdf-793f4f095f7a admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:34.925944 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d80312-b4dd-44f4-8cdf-793f4f095f7a admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:34.931348 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b52702e-16ac-444c-9241-49306021414b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:34.933939 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b52702e-16ac-444c-9241-49306021414b admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:34.934299 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b52702e-16ac-444c-9241-49306021414b admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:34.935084 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b52702e-16ac-444c-9241-49306021414b admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:34.935943 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b52702e-16ac-444c-9241-49306021414b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:34.938051 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b52702e-16ac-444c-9241-49306021414b admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:34.959897 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1003/1982] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:50:34.960900 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e2158-dfa9-457d-b792-0b0b5f8dc83c admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:34.973394 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e2158-dfa9-457d-b792-0b0b5f8dc83c admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:34.980755 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a75f3b-0180-481b-864d-6908a924e845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:34.981469 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a75f3b-0180-481b-864d-6908a924e845 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:34.982019 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a75f3b-0180-481b-864d-6908a924e845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:34.982385 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a75f3b-0180-481b-864d-6908a924e845 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:34.983096 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a75f3b-0180-481b-864d-6908a924e845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:34.984983 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a75f3b-0180-481b-864d-6908a924e845 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:34.998419 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 980/1983] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:50:35.002354 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b52702e-16ac-444c-9241-49306021414b admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:35.014077 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b52702e-16ac-444c-9241-49306021414b admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:35.020575 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78622d66-560d-4f19-bcad-68ee2b24498a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:35.021335 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78622d66-560d-4f19-bcad-68ee2b24498a admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:35.021620 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78622d66-560d-4f19-bcad-68ee2b24498a admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:35.021988 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78622d66-560d-4f19-bcad-68ee2b24498a admin admin] PATH_INFO: `/v3/projects/1652f269070f499286315f9614e2e204/users/63e2aa1b14d54dbf864ba886a2b64cff/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:35.044784 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78622d66-560d-4f19-bcad-68ee2b24498a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1652f269070f499286315f9614e2e204, user_id=63e2aa1b14d54dbf864ba886a2b64cff, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:35.047903 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78622d66-560d-4f19-bcad-68ee2b24498a admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:35.126737 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1004/1984] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:50:35 2022] PUT /identity/v3/projects/1652f269070f499286315f9614e2e204/users/63e2aa1b14d54dbf864ba886a2b64cff/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:50:35.141692 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d75e8-854e-4520-a723-63ac267cf0d3 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:35.142023 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d75e8-854e-4520-a723-63ac267cf0d3 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:35.142243 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d75e8-854e-4520-a723-63ac267cf0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:35.173219 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-044d75e8-854e-4520-a723-63ac267cf0d3 None None] MFA Rules not processed for user `63e2aa1b14d54dbf864ba886a2b64cff`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:35.174390 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-044d75e8-854e-4520-a723-63ac267cf0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:35.250614 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 981/1985] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:35.262565 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78622d66-560d-4f19-bcad-68ee2b24498a admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:35.264237 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-78622d66-560d-4f19-bcad-68ee2b24498a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:35.320591 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78622d66-560d-4f19-bcad-68ee2b24498a admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:35.324244 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cad70f-1856-4dfa-9763-72723ff6bcce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:35.324853 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cad70f-1856-4dfa-9763-72723ff6bcce service cinder] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:35.325135 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cad70f-1856-4dfa-9763-72723ff6bcce service cinder] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:35.325408 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cad70f-1856-4dfa-9763-72723ff6bcce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:35.335500 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cad70f-1856-4dfa-9763-72723ff6bcce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:35.337051 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cad70f-1856-4dfa-9763-72723ff6bcce service cinder] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:35.348986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1005/1986] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:50:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:50:45.173027 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d75e8-854e-4520-a723-63ac267cf0d3 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:45.240922 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d75e8-854e-4520-a723-63ac267cf0d3 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:45.247844 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34aa010-83ba-4c7e-89b9-272d4c8decf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:45.248504 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34aa010-83ba-4c7e-89b9-272d4c8decf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:45.248934 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34aa010-83ba-4c7e-89b9-272d4c8decf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:45.249217 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34aa010-83ba-4c7e-89b9-272d4c8decf8 admin admin] PATH_INFO: `/v3/users/63e2aa1b14d54dbf864ba886a2b64cff` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:45.250579 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34aa010-83ba-4c7e-89b9-272d4c8decf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63e2aa1b14d54dbf864ba886a2b64cff)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:45.252925 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34aa010-83ba-4c7e-89b9-272d4c8decf8 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:45.276593 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:45.342486 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:45.396360 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cad70f-1856-4dfa-9763-72723ff6bcce service cinder] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:45.399031 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-35cad70f-1856-4dfa-9763-72723ff6bcce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:45.414725 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cad70f-1856-4dfa-9763-72723ff6bcce service cinder] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:45.419812 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a8b03a-0f96-4e51-9262-25fc08df8e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:45.420255 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8b03a-0f96-4e51-9262-25fc08df8e95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:45.420571 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8b03a-0f96-4e51-9262-25fc08df8e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:45.421062 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8b03a-0f96-4e51-9262-25fc08df8e95 admin admin] PATH_INFO: `/v3/users/c46900e3bb3147a7840a76bef9268bf0` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:45.422207 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a8b03a-0f96-4e51-9262-25fc08df8e95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c46900e3bb3147a7840a76bef9268bf0)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:45.424419 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a8b03a-0f96-4e51-9262-25fc08df8e95 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:45.436596 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d34aa010-83ba-4c7e-89b9-272d4c8decf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63e2aa1b14d54dbf864ba886a2b64cff'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:45.477904 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d34aa010-83ba-4c7e-89b9-272d4c8decf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63e2aa1b14d54dbf864ba886a2b64cff'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:45.477904 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d34aa010-83ba-4c7e-89b9-272d4c8decf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63e2aa1b14d54dbf864ba886a2b64cff'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:45.478479 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d34aa010-83ba-4c7e-89b9-272d4c8decf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63e2aa1b14d54dbf864ba886a2b64cff'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:45.569625 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d34aa010-83ba-4c7e-89b9-272d4c8decf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63e2aa1b14d54dbf864ba886a2b64cff'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:45.579282 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 982/1987] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:50:45 2022] DELETE /identity/v3/users/63e2aa1b14d54dbf864ba886a2b64cff => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:50:45.582624 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d550b974-8206-4372-a6ab-eb83c83ee561 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:45.583135 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d550b974-8206-4372-a6ab-eb83c83ee561 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:45.583540 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d550b974-8206-4372-a6ab-eb83c83ee561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:45.623551 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-d550b974-8206-4372-a6ab-eb83c83ee561 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:45.624789 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d550b974-8206-4372-a6ab-eb83c83ee561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:45.653239 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-b6a8b03a-0f96-4e51-9262-25fc08df8e95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c46900e3bb3147a7840a76bef9268bf0'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:45.660835 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-b6a8b03a-0f96-4e51-9262-25fc08df8e95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c46900e3bb3147a7840a76bef9268bf0'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:45.686427 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-b6a8b03a-0f96-4e51-9262-25fc08df8e95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c46900e3bb3147a7840a76bef9268bf0'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:45.687221 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-b6a8b03a-0f96-4e51-9262-25fc08df8e95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c46900e3bb3147a7840a76bef9268bf0'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:45.702066 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 983/1988] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:50:45.712253 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d550b974-8206-4372-a6ab-eb83c83ee561 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:45.713681 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d550b974-8206-4372-a6ab-eb83c83ee561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:45.786599 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-b6a8b03a-0f96-4e51-9262-25fc08df8e95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c46900e3bb3147a7840a76bef9268bf0'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:45.792512 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d550b974-8206-4372-a6ab-eb83c83ee561 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:45.795356 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1006/1989] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:50:45 2022] DELETE /identity/v3/users/c46900e3bb3147a7840a76bef9268bf0 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:45.799596 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78539d-5388-4d7b-abda-397d9f48a8f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:45.799913 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7fb430-c01f-4a6a-8a6a-ae05737d0e8c None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:45.800173 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78539d-5388-4d7b-abda-397d9f48a8f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:45.800274 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7fb430-c01f-4a6a-8a6a-ae05737d0e8c None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:45.801165 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7fb430-c01f-4a6a-8a6a-ae05737d0e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:45.801295 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78539d-5388-4d7b-abda-397d9f48a8f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:45.801707 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78539d-5388-4d7b-abda-397d9f48a8f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:45.817690 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b78539d-5388-4d7b-abda-397d9f48a8f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:45.820223 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b78539d-5388-4d7b-abda-397d9f48a8f8 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:45.822907 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-5e7fb430-c01f-4a6a-8a6a-ae05737d0e8c None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:45.824397 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-5e7fb430-c01f-4a6a-8a6a-ae05737d0e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:45.834489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 984/1990] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:50:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:50:45.895248 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1007/1991] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:50:45.906027 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78539d-5388-4d7b-abda-397d9f48a8f8 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:45.908126 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-2b78539d-5388-4d7b-abda-397d9f48a8f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:45.920379 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78539d-5388-4d7b-abda-397d9f48a8f8 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:45.966089 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:45.977988 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0aaa25-b05c-44ed-818b-a710202329c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:45.982039 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0aaa25-b05c-44ed-818b-a710202329c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:45.982039 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0aaa25-b05c-44ed-818b-a710202329c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:45.982039 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0aaa25-b05c-44ed-818b-a710202329c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:45.991011 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0aaa25-b05c-44ed-818b-a710202329c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:45.992710 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0aaa25-b05c-44ed-818b-a710202329c3 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:46.003262 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 985/1992] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:50:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:50:46.391620 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7fb430-c01f-4a6a-8a6a-ae05737d0e8c None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:46.393216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-5e7fb430-c01f-4a6a-8a6a-ae05737d0e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:46.408568 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7fb430-c01f-4a6a-8a6a-ae05737d0e8c None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:46.416884 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89422a1-1db8-4aea-9945-94ecbb10d9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:46.417777 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89422a1-1db8-4aea-9945-94ecbb10d9b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:46.418275 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89422a1-1db8-4aea-9945-94ecbb10d9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:46.418794 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89422a1-1db8-4aea-9945-94ecbb10d9b0 admin admin] PATH_INFO: `/v3/projects/1652f269070f499286315f9614e2e204` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:46.420222 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89422a1-1db8-4aea-9945-94ecbb10d9b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1652f269070f499286315f9614e2e204)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:46.423099 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89422a1-1db8-4aea-9945-94ecbb10d9b0 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:46.565690 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0aaa25-b05c-44ed-818b-a710202329c3 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:46.567852 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-0c0aaa25-b05c-44ed-818b-a710202329c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:46.578491 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-c89422a1-1db8-4aea-9945-94ecbb10d9b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1652f269070f499286315f9614e2e204', 'request_id': 'req-c89422a1-1db8-4aea-9945-94ecbb10d9b0'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:46.581630 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0aaa25-b05c-44ed-818b-a710202329c3 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:46.585896 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-c89422a1-1db8-4aea-9945-94ecbb10d9b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1652f269070f499286315f9614e2e204', 'request_id': 'req-c89422a1-1db8-4aea-9945-94ecbb10d9b0'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:46.644863 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3472ff-9211-4a03-8cb8-0ed187afc82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:46.656222 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3472ff-9211-4a03-8cb8-0ed187afc82d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:46.656222 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3472ff-9211-4a03-8cb8-0ed187afc82d admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:46.656222 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3472ff-9211-4a03-8cb8-0ed187afc82d admin admin] PATH_INFO: `/v3/projects/4ca252947c74475d98ede7ec92ecc9c5` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:46.656222 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3472ff-9211-4a03-8cb8-0ed187afc82d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ca252947c74475d98ede7ec92ecc9c5)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:46.656222 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3472ff-9211-4a03-8cb8-0ed187afc82d admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:46.682691 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-c89422a1-1db8-4aea-9945-94ecbb10d9b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1652f269070f499286315f9614e2e204', 'request_id': 'req-c89422a1-1db8-4aea-9945-94ecbb10d9b0'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:46.689556 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:46.711222 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-c89422a1-1db8-4aea-9945-94ecbb10d9b0 admin admin] The token cache is being invalidate because project 1652f269070f499286315f9614e2e204 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:50:46.711584 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-c89422a1-1db8-4aea-9945-94ecbb10d9b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:46.720877 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-c89422a1-1db8-4aea-9945-94ecbb10d9b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:46.725932 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1008/1993] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:50:46 2022] DELETE /identity/v3/projects/1652f269070f499286315f9614e2e204 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:46.727344 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4976993-ff99-44e3-b144-8ac9b63dbfb9 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:46.727642 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4976993-ff99-44e3-b144-8ac9b63dbfb9 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:46.728240 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4976993-ff99-44e3-b144-8ac9b63dbfb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:46.770136 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1b3472ff-9211-4a03-8cb8-0ed187afc82d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ca252947c74475d98ede7ec92ecc9c5', 'request_id': 'req-1b3472ff-9211-4a03-8cb8-0ed187afc82d'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:46.772511 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-b4976993-ff99-44e3-b144-8ac9b63dbfb9 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:46.773778 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-b4976993-ff99-44e3-b144-8ac9b63dbfb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:46.801086 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1b3472ff-9211-4a03-8cb8-0ed187afc82d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ca252947c74475d98ede7ec92ecc9c5', 'request_id': 'req-1b3472ff-9211-4a03-8cb8-0ed187afc82d'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:46.868136 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1009/1994] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:46.875859 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ba34d-8dd6-4725-b1fb-a81d743b294e None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:46.876152 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ba34d-8dd6-4725-b1fb-a81d743b294e None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:46.876435 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ba34d-8dd6-4725-b1fb-a81d743b294e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:46.896616 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-780ba34d-8dd6-4725-b1fb-a81d743b294e None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:46.897808 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-780ba34d-8dd6-4725-b1fb-a81d743b294e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:46.905938 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1010/1995] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:50:46.910626 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ba34d-8dd6-4725-b1fb-a81d743b294e None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:46.919863 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ba34d-8dd6-4725-b1fb-a81d743b294e None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:46.924664 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ceebc-e987-4d37-b695-85280c75452e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:46.925303 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ceebc-e987-4d37-b695-85280c75452e admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:46.925597 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ceebc-e987-4d37-b695-85280c75452e admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:46.926047 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ceebc-e987-4d37-b695-85280c75452e admin admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:46.926741 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781ceebc-e987-4d37-b695-85280c75452e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:50:46.927398 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781ceebc-e987-4d37-b695-85280c75452e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:46.928283 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1b3472ff-9211-4a03-8cb8-0ed187afc82d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ca252947c74475d98ede7ec92ecc9c5', 'request_id': 'req-1b3472ff-9211-4a03-8cb8-0ed187afc82d'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:46.928984 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781ceebc-e987-4d37-b695-85280c75452e admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:46.940403 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1b3472ff-9211-4a03-8cb8-0ed187afc82d admin admin] The token cache is being invalidate because project 4ca252947c74475d98ede7ec92ecc9c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:50:46.940777 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1b3472ff-9211-4a03-8cb8-0ed187afc82d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:46.941566 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1b3472ff-9211-4a03-8cb8-0ed187afc82d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:46.943147 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 986/1996] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:50:46 2022] DELETE /identity/v3/projects/4ca252947c74475d98ede7ec92ecc9c5 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:46.947070 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1011/1997] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:50:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:50:46.947420 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3472ff-9211-4a03-8cb8-0ed187afc82d admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:46.948057 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ceebc-e987-4d37-b695-85280c75452e admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:46.948734 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1b3472ff-9211-4a03-8cb8-0ed187afc82d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:46.949633 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-781ceebc-e987-4d37-b695-85280c75452e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:46.964296 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3472ff-9211-4a03-8cb8-0ed187afc82d admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:46.965314 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ceebc-e987-4d37-b695-85280c75452e admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:46.970022 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e41e08-fad6-4b03-adae-c5002cd0403c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:46.970278 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a528cef-c7f8-4446-a621-589680293fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:46.970616 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e41e08-fad6-4b03-adae-c5002cd0403c admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:46.970899 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e41e08-fad6-4b03-adae-c5002cd0403c admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:46.971177 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e41e08-fad6-4b03-adae-c5002cd0403c admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:46.971851 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a528cef-c7f8-4446-a621-589680293fa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:46.971967 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e41e08-fad6-4b03-adae-c5002cd0403c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:46.973053 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a528cef-c7f8-4446-a621-589680293fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:46.974508 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a528cef-c7f8-4446-a621-589680293fa7 admin admin] PATH_INFO: `/v3/users/56d217d287a9435896c23e56b762ea4a` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:46.974882 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e41e08-fad6-4b03-adae-c5002cd0403c admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:46.976026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a528cef-c7f8-4446-a621-589680293fa7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56d217d287a9435896c23e56b762ea4a)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:46.978834 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a528cef-c7f8-4446-a621-589680293fa7 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:47.047480 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1012/1998] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:50:46 2022] POST /identity/v3/projects => generated 371 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:50:47.057016 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e41e08-fad6-4b03-adae-c5002cd0403c admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:47.129825 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e41e08-fad6-4b03-adae-c5002cd0403c admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:47.136932 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9daf8-f315-4c49-a8c2-f16268e4a5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:47.137557 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9daf8-f315-4c49-a8c2-f16268e4a5b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:47.137826 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9daf8-f315-4c49-a8c2-f16268e4a5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:47.138096 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9daf8-f315-4c49-a8c2-f16268e4a5b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:47.138719 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d9daf8-f315-4c49-a8c2-f16268e4a5b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:47.140648 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d9daf8-f315-4c49-a8c2-f16268e4a5b0 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:47.178861 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7a528cef-c7f8-4446-a621-589680293fa7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56d217d287a9435896c23e56b762ea4a'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:47.192249 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7a528cef-c7f8-4446-a621-589680293fa7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56d217d287a9435896c23e56b762ea4a'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:47.194063 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7a528cef-c7f8-4446-a621-589680293fa7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56d217d287a9435896c23e56b762ea4a'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:47.197759 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:47.200817 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7a528cef-c7f8-4446-a621-589680293fa7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56d217d287a9435896c23e56b762ea4a'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:47.223287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1013/1999] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:50:47 2022] POST /identity/v3/users => generated 286 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:47.224655 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9daf8-f315-4c49-a8c2-f16268e4a5b0 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:47.226852 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-b4d9daf8-f315-4c49-a8c2-f16268e4a5b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:47.240950 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9daf8-f315-4c49-a8c2-f16268e4a5b0 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:47.245609 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd7124-e763-4bc2-aae9-5bc8ae149db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:47.246224 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd7124-e763-4bc2-aae9-5bc8ae149db3 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:47.246515 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd7124-e763-4bc2-aae9-5bc8ae149db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:47.246805 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd7124-e763-4bc2-aae9-5bc8ae149db3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:47.247521 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd7124-e763-4bc2-aae9-5bc8ae149db3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:47.249124 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd7124-e763-4bc2-aae9-5bc8ae149db3 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:47.259889 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1014/2000] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:50:47.263758 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd7124-e763-4bc2-aae9-5bc8ae149db3 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:47.274384 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd7124-e763-4bc2-aae9-5bc8ae149db3 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:47.279117 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44a145-9521-4138-a70f-d4a23c64a562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:47.279770 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44a145-9521-4138-a70f-d4a23c64a562 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:47.280051 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44a145-9521-4138-a70f-d4a23c64a562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:47.280324 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44a145-9521-4138-a70f-d4a23c64a562 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:47.280979 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc44a145-9521-4138-a70f-d4a23c64a562 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:47.282441 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7a528cef-c7f8-4446-a621-589680293fa7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56d217d287a9435896c23e56b762ea4a'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:47.283572 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc44a145-9521-4138-a70f-d4a23c64a562 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:47.296080 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 987/2001] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:50:46 2022] DELETE /identity/v3/users/56d217d287a9435896c23e56b762ea4a => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:50:47.301019 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1015/2002] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:50:47.318621 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a528cef-c7f8-4446-a621-589680293fa7 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:47.395074 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a528cef-c7f8-4446-a621-589680293fa7 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:47.399512 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6cc347-8286-403f-aaaa-47ea7884261e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:47.400182 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6cc347-8286-403f-aaaa-47ea7884261e admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:47.400461 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6cc347-8286-403f-aaaa-47ea7884261e admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:47.400735 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6cc347-8286-403f-aaaa-47ea7884261e admin admin] PATH_INFO: `/v3/projects/f20e08e1248d44329bdb93e6e3f09796/users/1375c2dded8b4ea1a3ae9730b2525b5b/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:47.417396 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6cc347-8286-403f-aaaa-47ea7884261e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f20e08e1248d44329bdb93e6e3f09796, user_id=1375c2dded8b4ea1a3ae9730b2525b5b, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:47.420117 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6cc347-8286-403f-aaaa-47ea7884261e admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:47.490267 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 988/2003] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:50:47 2022] PUT /identity/v3/projects/f20e08e1248d44329bdb93e6e3f09796/users/1375c2dded8b4ea1a3ae9730b2525b5b/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:47.500610 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7e421-2036-4292-a5ec-31a6ca5c1140 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:47.501099 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7e421-2036-4292-a5ec-31a6ca5c1140 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:47.501180 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7e421-2036-4292-a5ec-31a6ca5c1140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:47.530740 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-bdf7e421-2036-4292-a5ec-31a6ca5c1140 None None] MFA Rules not processed for user `1375c2dded8b4ea1a3ae9730b2525b5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:47.532928 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-bdf7e421-2036-4292-a5ec-31a6ca5c1140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:47.554459 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:47.602766 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1016/2004] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:47.615650 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6cc347-8286-403f-aaaa-47ea7884261e admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:47.617409 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ac6cc347-8286-403f-aaaa-47ea7884261e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:47.699131 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6cc347-8286-403f-aaaa-47ea7884261e admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:47.702812 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f905929-95ed-4dab-a067-82f133c515b3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b697e4847b13496d98a543675f5df1c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:47.703455 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f905929-95ed-4dab-a067-82f133c515b3 service glance] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:47.703736 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f905929-95ed-4dab-a067-82f133c515b3 service glance] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:47.704014 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f905929-95ed-4dab-a067-82f133c515b3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:47.713154 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f905929-95ed-4dab-a067-82f133c515b3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:47.714689 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f905929-95ed-4dab-a067-82f133c515b3 service glance] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:47.724209 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 989/2005] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:50:47 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:50:47.780521 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf7e421-2036-4292-a5ec-31a6ca5c1140 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:47.782374 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-bdf7e421-2036-4292-a5ec-31a6ca5c1140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:47.900451 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf7e421-2036-4292-a5ec-31a6ca5c1140 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:47.907360 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21162bf4-071d-45a8-ac0a-48e11e477239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:47.907360 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21162bf4-071d-45a8-ac0a-48e11e477239 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:47.907360 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21162bf4-071d-45a8-ac0a-48e11e477239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:47.907360 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21162bf4-071d-45a8-ac0a-48e11e477239 admin admin] PATH_INFO: `/v3/projects/acf8de0832624632906cd4d80337a83c` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:47.909348 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21162bf4-071d-45a8-ac0a-48e11e477239 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acf8de0832624632906cd4d80337a83c)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:47.911688 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21162bf4-071d-45a8-ac0a-48e11e477239 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:48.047360 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-21162bf4-071d-45a8-ac0a-48e11e477239 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acf8de0832624632906cd4d80337a83c', 'request_id': 'req-21162bf4-071d-45a8-ac0a-48e11e477239'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:48.062480 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-21162bf4-071d-45a8-ac0a-48e11e477239 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acf8de0832624632906cd4d80337a83c', 'request_id': 'req-21162bf4-071d-45a8-ac0a-48e11e477239'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:48.139484 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-21162bf4-071d-45a8-ac0a-48e11e477239 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acf8de0832624632906cd4d80337a83c', 'request_id': 'req-21162bf4-071d-45a8-ac0a-48e11e477239'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:48.152058 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-21162bf4-071d-45a8-ac0a-48e11e477239 admin admin] The token cache is being invalidate because project acf8de0832624632906cd4d80337a83c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:50:48.152346 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-21162bf4-071d-45a8-ac0a-48e11e477239 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:48.153006 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-21162bf4-071d-45a8-ac0a-48e11e477239 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:48.154540 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1017/2006] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:50:47 2022] DELETE /identity/v3/projects/acf8de0832624632906cd4d80337a83c => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:48.158473 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f905929-95ed-4dab-a067-82f133c515b3 service glance] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:48.169621 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f905929-95ed-4dab-a067-82f133c515b3 service glance] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:48.221430 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:48.228328 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21162bf4-071d-45a8-ac0a-48e11e477239 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:48.230043 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-21162bf4-071d-45a8-ac0a-48e11e477239 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:48.237331 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c921dd53-abf1-45b3-9685-03f257b385bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:48.238788 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921dd53-abf1-45b3-9685-03f257b385bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:48.239064 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921dd53-abf1-45b3-9685-03f257b385bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:48.239351 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921dd53-abf1-45b3-9685-03f257b385bd admin admin] PATH_INFO: `/v3/users/56215af8bd2b44c8a9509bc4d904c0e5` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:48.240582 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c921dd53-abf1-45b3-9685-03f257b385bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=56215af8bd2b44c8a9509bc4d904c0e5)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:48.242927 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c921dd53-abf1-45b3-9685-03f257b385bd admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:48.276129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21162bf4-071d-45a8-ac0a-48e11e477239 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:48.283121 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d2c20c-aa21-4b60-8b42-8c075e8aa9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:48.283121 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d2c20c-aa21-4b60-8b42-8c075e8aa9f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:48.283734 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d2c20c-aa21-4b60-8b42-8c075e8aa9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:48.283734 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d2c20c-aa21-4b60-8b42-8c075e8aa9f1 admin admin] PATH_INFO: `/v3/users/40b84af5a0cb410f8bb6fe48fb56e66a` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:48.285595 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d2c20c-aa21-4b60-8b42-8c075e8aa9f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40b84af5a0cb410f8bb6fe48fb56e66a)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:48.287659 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d2c20c-aa21-4b60-8b42-8c075e8aa9f1 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:48.430506 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c921dd53-abf1-45b3-9685-03f257b385bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56215af8bd2b44c8a9509bc4d904c0e5'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:48.444437 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c921dd53-abf1-45b3-9685-03f257b385bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56215af8bd2b44c8a9509bc4d904c0e5'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:48.445144 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c921dd53-abf1-45b3-9685-03f257b385bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56215af8bd2b44c8a9509bc4d904c0e5'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:48.452381 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c921dd53-abf1-45b3-9685-03f257b385bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56215af8bd2b44c8a9509bc4d904c0e5'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:48.495120 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:48.506416 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-b0d2c20c-aa21-4b60-8b42-8c075e8aa9f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40b84af5a0cb410f8bb6fe48fb56e66a'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:48.511418 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-b0d2c20c-aa21-4b60-8b42-8c075e8aa9f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40b84af5a0cb410f8bb6fe48fb56e66a'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:48.525066 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-b0d2c20c-aa21-4b60-8b42-8c075e8aa9f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40b84af5a0cb410f8bb6fe48fb56e66a'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:48.525912 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-b0d2c20c-aa21-4b60-8b42-8c075e8aa9f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40b84af5a0cb410f8bb6fe48fb56e66a'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:48.544953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c921dd53-abf1-45b3-9685-03f257b385bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56215af8bd2b44c8a9509bc4d904c0e5'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:48.551578 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 990/2007] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:50:48 2022] DELETE /identity/v3/users/56215af8bd2b44c8a9509bc4d904c0e5 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:48.629248 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-b0d2c20c-aa21-4b60-8b42-8c075e8aa9f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40b84af5a0cb410f8bb6fe48fb56e66a'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:48.650231 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1018/2008] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:50:48 2022] DELETE /identity/v3/users/40b84af5a0cb410f8bb6fe48fb56e66a => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:48.654514 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df203cfd-3cf7-4076-b0c7-ee5bbf24cfeb None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:48.655850 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df203cfd-3cf7-4076-b0c7-ee5bbf24cfeb None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:48.655850 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df203cfd-3cf7-4076-b0c7-ee5bbf24cfeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:48.679007 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-df203cfd-3cf7-4076-b0c7-ee5bbf24cfeb None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:48.679007 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-df203cfd-3cf7-4076-b0c7-ee5bbf24cfeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:48.742535 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 991/2009] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:48.752743 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d2c20c-aa21-4b60-8b42-8c075e8aa9f1 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:48.754729 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-b0d2c20c-aa21-4b60-8b42-8c075e8aa9f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:48.815191 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d2c20c-aa21-4b60-8b42-8c075e8aa9f1 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:48.819952 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723a2a53-a782-447a-afdd-5d81f496462a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:48.820453 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a2a53-a782-447a-afdd-5d81f496462a service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:48.820876 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a2a53-a782-447a-afdd-5d81f496462a service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:48.821141 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a2a53-a782-447a-afdd-5d81f496462a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:48.834505 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723a2a53-a782-447a-afdd-5d81f496462a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:48.836040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723a2a53-a782-447a-afdd-5d81f496462a service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:48.848752 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1019/2010] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:50:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:50:49.181079 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df203cfd-3cf7-4076-b0c7-ee5bbf24cfeb None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:49.182496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-df203cfd-3cf7-4076-b0c7-ee5bbf24cfeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:49.194879 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df203cfd-3cf7-4076-b0c7-ee5bbf24cfeb None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:49.200628 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2171bbf9-7be9-43f8-ad3c-1bed20f21dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:49.200628 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171bbf9-7be9-43f8-ad3c-1bed20f21dec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:49.200628 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171bbf9-7be9-43f8-ad3c-1bed20f21dec admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:49.201130 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171bbf9-7be9-43f8-ad3c-1bed20f21dec admin admin] PATH_INFO: `/v3/projects/f153d879299443148eddecb8269c117d` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:49.202743 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2171bbf9-7be9-43f8-ad3c-1bed20f21dec admin admin] RBAC: Authorizing `identity:delete_project(project_id=f153d879299443148eddecb8269c117d)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:49.204753 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2171bbf9-7be9-43f8-ad3c-1bed20f21dec admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:49.405953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-2171bbf9-7be9-43f8-ad3c-1bed20f21dec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f153d879299443148eddecb8269c117d', 'request_id': 'req-2171bbf9-7be9-43f8-ad3c-1bed20f21dec'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:49.412757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-2171bbf9-7be9-43f8-ad3c-1bed20f21dec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f153d879299443148eddecb8269c117d', 'request_id': 'req-2171bbf9-7be9-43f8-ad3c-1bed20f21dec'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:49.474382 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723a2a53-a782-447a-afdd-5d81f496462a service neutron] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:49.475889 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-723a2a53-a782-447a-afdd-5d81f496462a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:49.494938 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723a2a53-a782-447a-afdd-5d81f496462a service neutron] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:49.509220 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-2171bbf9-7be9-43f8-ad3c-1bed20f21dec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f153d879299443148eddecb8269c117d', 'request_id': 'req-2171bbf9-7be9-43f8-ad3c-1bed20f21dec'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:49.511967 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:49.522820 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-2171bbf9-7be9-43f8-ad3c-1bed20f21dec admin admin] The token cache is being invalidate because project f153d879299443148eddecb8269c117d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:50:49.523116 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-2171bbf9-7be9-43f8-ad3c-1bed20f21dec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:49.523842 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-2171bbf9-7be9-43f8-ad3c-1bed20f21dec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:49.526017 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 992/2011] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:50:49 2022] DELETE /identity/v3/projects/f153d879299443148eddecb8269c117d => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:49.531082 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a439b0d-0165-4bc1-912d-a69b100e417a None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:49.531535 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a439b0d-0165-4bc1-912d-a69b100e417a None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:49.531668 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a439b0d-0165-4bc1-912d-a69b100e417a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:49.554066 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-8a439b0d-0165-4bc1-912d-a69b100e417a None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:49.555392 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-8a439b0d-0165-4bc1-912d-a69b100e417a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:49.564591 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25987ef2-8708-4c0f-88b6-8044a657d8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:49.565466 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25987ef2-8708-4c0f-88b6-8044a657d8ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:49.566008 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25987ef2-8708-4c0f-88b6-8044a657d8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:49.567028 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25987ef2-8708-4c0f-88b6-8044a657d8ac admin admin] PATH_INFO: `/v3/projects/2364d42ae14344caa1abfbab7816d28b` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:49.568715 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25987ef2-8708-4c0f-88b6-8044a657d8ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=2364d42ae14344caa1abfbab7816d28b)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:49.571114 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25987ef2-8708-4c0f-88b6-8044a657d8ac admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:49.634165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 993/2012] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:49.642147 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22998bd0-cd9f-41b2-a088-61ae9191c336 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:49.642434 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22998bd0-cd9f-41b2-a088-61ae9191c336 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:49.642708 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22998bd0-cd9f-41b2-a088-61ae9191c336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:49.664957 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-22998bd0-cd9f-41b2-a088-61ae9191c336 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:49.667610 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-22998bd0-cd9f-41b2-a088-61ae9191c336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:49.677310 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 994/2013] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:50:49.681803 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22998bd0-cd9f-41b2-a088-61ae9191c336 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:49.691042 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22998bd0-cd9f-41b2-a088-61ae9191c336 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:49.697788 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2fd206-4fcb-4ec8-b858-d74cdb90d1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:49.698644 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2fd206-4fcb-4ec8-b858-d74cdb90d1a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:49.698928 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2fd206-4fcb-4ec8-b858-d74cdb90d1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:49.704296 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2fd206-4fcb-4ec8-b858-d74cdb90d1a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:49.708536 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2fd206-4fcb-4ec8-b858-d74cdb90d1a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:50:49.708536 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2fd206-4fcb-4ec8-b858-d74cdb90d1a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:49.708536 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2fd206-4fcb-4ec8-b858-d74cdb90d1a3 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:49.717659 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-25987ef2-8708-4c0f-88b6-8044a657d8ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2364d42ae14344caa1abfbab7816d28b', 'request_id': 'req-25987ef2-8708-4c0f-88b6-8044a657d8ac'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:49.731520 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 995/2014] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:50:49.731847 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-25987ef2-8708-4c0f-88b6-8044a657d8ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2364d42ae14344caa1abfbab7816d28b', 'request_id': 'req-25987ef2-8708-4c0f-88b6-8044a657d8ac'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:49.731847 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:49.741216 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2fd206-4fcb-4ec8-b858-d74cdb90d1a3 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:49.752479 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2fd206-4fcb-4ec8-b858-d74cdb90d1a3 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:49.817697 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a103b-8fe9-4cf6-9c3f-965b1713d048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:49.818302 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a103b-8fe9-4cf6-9c3f-965b1713d048 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:49.818670 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a103b-8fe9-4cf6-9c3f-965b1713d048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:49.818882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a103b-8fe9-4cf6-9c3f-965b1713d048 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:49.819877 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5a103b-8fe9-4cf6-9c3f-965b1713d048 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:49.820029 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-25987ef2-8708-4c0f-88b6-8044a657d8ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2364d42ae14344caa1abfbab7816d28b', 'request_id': 'req-25987ef2-8708-4c0f-88b6-8044a657d8ac'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:49.823337 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5a103b-8fe9-4cf6-9c3f-965b1713d048 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:49.832909 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-25987ef2-8708-4c0f-88b6-8044a657d8ac admin admin] The token cache is being invalidate because project 2364d42ae14344caa1abfbab7816d28b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:50:49.833218 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-25987ef2-8708-4c0f-88b6-8044a657d8ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:49.833953 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-25987ef2-8708-4c0f-88b6-8044a657d8ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:49.835692 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1020/2015] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:50:49 2022] DELETE /identity/v3/projects/2364d42ae14344caa1abfbab7816d28b => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:50:49.839744 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25987ef2-8708-4c0f-88b6-8044a657d8ac admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:49.841130 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-25987ef2-8708-4c0f-88b6-8044a657d8ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:49.855782 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25987ef2-8708-4c0f-88b6-8044a657d8ac admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:49.860468 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed0e6f6-7c9b-4c5f-90f7-c0275ac0291f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:49.861101 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed0e6f6-7c9b-4c5f-90f7-c0275ac0291f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:49.861377 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed0e6f6-7c9b-4c5f-90f7-c0275ac0291f admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:49.861645 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed0e6f6-7c9b-4c5f-90f7-c0275ac0291f admin admin] PATH_INFO: `/v3/users/27d21574ca16490581dfe9edce146952` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:49.863943 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed0e6f6-7c9b-4c5f-90f7-c0275ac0291f admin admin] RBAC: Authorizing `identity:delete_user(user_id=27d21574ca16490581dfe9edce146952)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:49.866774 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed0e6f6-7c9b-4c5f-90f7-c0275ac0291f admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:49.920326 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 996/2016] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:50:49 2022] POST /identity/v3/projects => generated 371 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:50:49.926179 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a103b-8fe9-4cf6-9c3f-965b1713d048 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:49.928533 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-0d5a103b-8fe9-4cf6-9c3f-965b1713d048 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:49.998693 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a103b-8fe9-4cf6-9c3f-965b1713d048 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:50.003310 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb785e3-35ba-4f77-a1dd-e6146b4ef81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:50.004057 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb785e3-35ba-4f77-a1dd-e6146b4ef81d admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:50.004505 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb785e3-35ba-4f77-a1dd-e6146b4ef81d admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:50.004778 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb785e3-35ba-4f77-a1dd-e6146b4ef81d admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:50.005399 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb785e3-35ba-4f77-a1dd-e6146b4ef81d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:50.007487 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb785e3-35ba-4f77-a1dd-e6146b4ef81d admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:50.018628 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:50.031338 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-3ed0e6f6-7c9b-4c5f-90f7-c0275ac0291f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27d21574ca16490581dfe9edce146952'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:50.039877 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-3ed0e6f6-7c9b-4c5f-90f7-c0275ac0291f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27d21574ca16490581dfe9edce146952'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:50.052389 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-3ed0e6f6-7c9b-4c5f-90f7-c0275ac0291f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27d21574ca16490581dfe9edce146952'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:50.053255 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-3ed0e6f6-7c9b-4c5f-90f7-c0275ac0291f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27d21574ca16490581dfe9edce146952'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:50.103519 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 997/2017] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:50:49 2022] POST /identity/v3/users => generated 286 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:50.104857 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb785e3-35ba-4f77-a1dd-e6146b4ef81d admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:50.106497 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-7fb785e3-35ba-4f77-a1dd-e6146b4ef81d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:50.111087 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-3ed0e6f6-7c9b-4c5f-90f7-c0275ac0291f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27d21574ca16490581dfe9edce146952'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:50.122645 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1021/2018] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:50:49 2022] DELETE /identity/v3/users/27d21574ca16490581dfe9edce146952 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:50:50.123717 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb785e3-35ba-4f77-a1dd-e6146b4ef81d admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:50.204979 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8869df2-c01b-4de9-b0dc-b71e1eb27597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:50.205574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8869df2-c01b-4de9-b0dc-b71e1eb27597 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:50.205855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8869df2-c01b-4de9-b0dc-b71e1eb27597 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:50.206127 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8869df2-c01b-4de9-b0dc-b71e1eb27597 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:50.206843 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8869df2-c01b-4de9-b0dc-b71e1eb27597 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:50.208533 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8869df2-c01b-4de9-b0dc-b71e1eb27597 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:50.219669 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 998/2019] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 12:50:50.224758 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed0e6f6-7c9b-4c5f-90f7-c0275ac0291f admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:50.246641 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed0e6f6-7c9b-4c5f-90f7-c0275ac0291f admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:50.256070 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61977470-9136-4224-9473-33cbc9fddf29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:50.256904 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61977470-9136-4224-9473-33cbc9fddf29 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:50.257331 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61977470-9136-4224-9473-33cbc9fddf29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:50.257666 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61977470-9136-4224-9473-33cbc9fddf29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:50.258475 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61977470-9136-4224-9473-33cbc9fddf29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:50.260234 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61977470-9136-4224-9473-33cbc9fddf29 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:50.272023 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1022/2020] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:50:50.275775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8869df2-c01b-4de9-b0dc-b71e1eb27597 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:50.286641 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8869df2-c01b-4de9-b0dc-b71e1eb27597 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:50.291476 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d25fce-5afc-4bd8-903d-f695d020df0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:50.292144 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d25fce-5afc-4bd8-903d-f695d020df0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:50.292424 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d25fce-5afc-4bd8-903d-f695d020df0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:50.292695 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d25fce-5afc-4bd8-903d-f695d020df0a admin admin] PATH_INFO: `/v3/projects/eb983cfa7dfe47b19a0a2d5ba652dabb/users/78b322344df24e8ca513c19361b79257/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:50.309167 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d25fce-5afc-4bd8-903d-f695d020df0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb983cfa7dfe47b19a0a2d5ba652dabb, user_id=78b322344df24e8ca513c19361b79257, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:50.311981 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d25fce-5afc-4bd8-903d-f695d020df0a admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:50.431594 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 999/2021] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:50:50 2022] PUT /identity/v3/projects/eb983cfa7dfe47b19a0a2d5ba652dabb/users/78b322344df24e8ca513c19361b79257/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:50.437137 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6a984-9823-4772-b4ab-fe4dec0a6bdc None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:50.437626 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6a984-9823-4772-b4ab-fe4dec0a6bdc None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:50.438006 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6a984-9823-4772-b4ab-fe4dec0a6bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:50.476652 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-e3f6a984-9823-4772-b4ab-fe4dec0a6bdc None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:50.483515 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-e3f6a984-9823-4772-b4ab-fe4dec0a6bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:50.561304 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1023/2022] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:50.571377 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d25fce-5afc-4bd8-903d-f695d020df0a admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:50.573090 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-68d25fce-5afc-4bd8-903d-f695d020df0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:50.644831 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d25fce-5afc-4bd8-903d-f695d020df0a admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:50.651960 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b71b84-fa97-4f58-9f7c-4303e48f2619 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:50.652729 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b71b84-fa97-4f58-9f7c-4303e48f2619 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:50.653207 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b71b84-fa97-4f58-9f7c-4303e48f2619 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:50.653540 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b71b84-fa97-4f58-9f7c-4303e48f2619 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:50.665600 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b71b84-fa97-4f58-9f7c-4303e48f2619 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:50.667514 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b71b84-fa97-4f58-9f7c-4303e48f2619 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:50.675081 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f6a984-9823-4772-b4ab-fe4dec0a6bdc None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:50.677310 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-e3f6a984-9823-4772-b4ab-fe4dec0a6bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:50.684748 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1000/2023] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:50:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:50:50.701869 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f6a984-9823-4772-b4ab-fe4dec0a6bdc None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:50.747020 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7199e850-6c0d-475f-8826-08b0d670c55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:50.747947 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199e850-6c0d-475f-8826-08b0d670c55d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:50.748426 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199e850-6c0d-475f-8826-08b0d670c55d admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:50.748909 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199e850-6c0d-475f-8826-08b0d670c55d admin admin] PATH_INFO: `/v3/projects/5c9d5ae85ee042a28915126e0e04b9d4` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:50.751820 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7199e850-6c0d-475f-8826-08b0d670c55d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c9d5ae85ee042a28915126e0e04b9d4)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:50.755976 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7199e850-6c0d-475f-8826-08b0d670c55d admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:50.853292 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b71b84-fa97-4f58-9f7c-4303e48f2619 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:50.857116 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-93b71b84-fa97-4f58-9f7c-4303e48f2619 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:50.875390 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b71b84-fa97-4f58-9f7c-4303e48f2619 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:50.886024 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454fc6fe-8a69-420e-8226-56459feedc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:50.886864 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fc6fe-8a69-420e-8226-56459feedc50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:50.887448 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fc6fe-8a69-420e-8226-56459feedc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:50.887762 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fc6fe-8a69-420e-8226-56459feedc50 admin admin] PATH_INFO: `/v3/users/c45303e113f34b3e8d710057074eaafe` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:50.889046 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454fc6fe-8a69-420e-8226-56459feedc50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c45303e113f34b3e8d710057074eaafe)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:50.891276 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454fc6fe-8a69-420e-8226-56459feedc50 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:50.961172 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:50.965166 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-7199e850-6c0d-475f-8826-08b0d670c55d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c9d5ae85ee042a28915126e0e04b9d4', 'request_id': 'req-7199e850-6c0d-475f-8826-08b0d670c55d'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:50.971303 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-7199e850-6c0d-475f-8826-08b0d670c55d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c9d5ae85ee042a28915126e0e04b9d4', 'request_id': 'req-7199e850-6c0d-475f-8826-08b0d670c55d'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:51.025487 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-7199e850-6c0d-475f-8826-08b0d670c55d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c9d5ae85ee042a28915126e0e04b9d4', 'request_id': 'req-7199e850-6c0d-475f-8826-08b0d670c55d'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:51.036922 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-7199e850-6c0d-475f-8826-08b0d670c55d admin admin] The token cache is being invalidate because project 5c9d5ae85ee042a28915126e0e04b9d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:50:51.037142 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-7199e850-6c0d-475f-8826-08b0d670c55d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:51.037902 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-7199e850-6c0d-475f-8826-08b0d670c55d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:51.039729 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1024/2024] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:50:50 2022] DELETE /identity/v3/projects/5c9d5ae85ee042a28915126e0e04b9d4 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:50:51.044316 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9b381-fb6e-4baa-84a8-3461cc98b2db None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:51.044601 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9b381-fb6e-4baa-84a8-3461cc98b2db None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:51.044882 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9b381-fb6e-4baa-84a8-3461cc98b2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:51.067161 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-d8a9b381-fb6e-4baa-84a8-3461cc98b2db None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:51.068420 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-d8a9b381-fb6e-4baa-84a8-3461cc98b2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:51.109112 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-454fc6fe-8a69-420e-8226-56459feedc50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c45303e113f34b3e8d710057074eaafe'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:51.124645 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:51.126824 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-454fc6fe-8a69-420e-8226-56459feedc50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c45303e113f34b3e8d710057074eaafe'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:51.127594 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-454fc6fe-8a69-420e-8226-56459feedc50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c45303e113f34b3e8d710057074eaafe'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:51.137076 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-454fc6fe-8a69-420e-8226-56459feedc50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c45303e113f34b3e8d710057074eaafe'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:51.169080 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1025/2025] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:50:51.177710 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4bdb8-0e3c-4151-b256-da27ba5613c2 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:51.177993 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4bdb8-0e3c-4151-b256-da27ba5613c2 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:51.178265 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4bdb8-0e3c-4151-b256-da27ba5613c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:51.196511 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-454fc6fe-8a69-420e-8226-56459feedc50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c45303e113f34b3e8d710057074eaafe'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:51.197873 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-bbe4bdb8-0e3c-4151-b256-da27ba5613c2 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:51.199243 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-bbe4bdb8-0e3c-4151-b256-da27ba5613c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:51.202892 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1001/2026] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:50:50 2022] DELETE /identity/v3/users/c45303e113f34b3e8d710057074eaafe => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:51.208042 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b16eb-7369-401f-b5d9-330f4d69b086 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:51.208333 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b16eb-7369-401f-b5d9-330f4d69b086 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:51.208612 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b16eb-7369-401f-b5d9-330f4d69b086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:51.210842 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1026/2027] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:50:51.219265 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe4bdb8-0e3c-4151-b256-da27ba5613c2 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:51.230518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe4bdb8-0e3c-4151-b256-da27ba5613c2 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:51.239744 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-a89b16eb-7369-401f-b5d9-330f4d69b086 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:51.241190 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a89b16eb-7369-401f-b5d9-330f4d69b086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:51.323720 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1002/2028] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:50:51.327488 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0465689a-db8c-41cd-a8f4-223f2562e972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:51.328128 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0465689a-db8c-41cd-a8f4-223f2562e972 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:51.328406 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0465689a-db8c-41cd-a8f4-223f2562e972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:51.328674 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0465689a-db8c-41cd-a8f4-223f2562e972 admin admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:51.329328 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0465689a-db8c-41cd-a8f4-223f2562e972 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:50:51.329632 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0465689a-db8c-41cd-a8f4-223f2562e972 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:51.331294 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0465689a-db8c-41cd-a8f4-223f2562e972 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:51.342436 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b16eb-7369-401f-b5d9-330f4d69b086 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:51.342436 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a89b16eb-7369-401f-b5d9-330f4d69b086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:51.346355 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1027/2029] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:50:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:50:51.353004 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0465689a-db8c-41cd-a8f4-223f2562e972 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:51.368956 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0465689a-db8c-41cd-a8f4-223f2562e972 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:51.380373 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737adf0-bc76-4256-a6dd-6e7796873b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:51.381105 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737adf0-bc76-4256-a6dd-6e7796873b9b admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:51.381456 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737adf0-bc76-4256-a6dd-6e7796873b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:51.381842 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737adf0-bc76-4256-a6dd-6e7796873b9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:51.382586 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b737adf0-bc76-4256-a6dd-6e7796873b9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:51.384662 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b737adf0-bc76-4256-a6dd-6e7796873b9b admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:51.423957 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b16eb-7369-401f-b5d9-330f4d69b086 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:51.427744 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805663f2-6648-4315-a042-764eaa2a7e45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:51.428572 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805663f2-6648-4315-a042-764eaa2a7e45 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:51.428849 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805663f2-6648-4315-a042-764eaa2a7e45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:51.429151 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805663f2-6648-4315-a042-764eaa2a7e45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:51.440023 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805663f2-6648-4315-a042-764eaa2a7e45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:51.442010 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805663f2-6648-4315-a042-764eaa2a7e45 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:51.451553 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1003/2030] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:50:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:50:51.505592 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1028/2031] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:50:51 2022] POST /identity/v3/projects => generated 387 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:51.509496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805663f2-6648-4315-a042-764eaa2a7e45 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:51.521710 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805663f2-6648-4315-a042-764eaa2a7e45 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:51.587977 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258aa126-30b0-420e-8a0b-a10c35638311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:51.588836 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258aa126-30b0-420e-8a0b-a10c35638311 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:51.589046 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258aa126-30b0-420e-8a0b-a10c35638311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:51.589392 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258aa126-30b0-420e-8a0b-a10c35638311 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:51.590169 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258aa126-30b0-420e-8a0b-a10c35638311 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:51.592372 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258aa126-30b0-420e-8a0b-a10c35638311 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:51.726214 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1004/2032] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:50:51 2022] POST /identity/v3/users => generated 294 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:51.728208 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737adf0-bc76-4256-a6dd-6e7796873b9b admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:51.748933 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737adf0-bc76-4256-a6dd-6e7796873b9b admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:51.754890 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc151201-7f31-4373-8ea5-d35d28217538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:51.755552 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc151201-7f31-4373-8ea5-d35d28217538 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:51.755838 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc151201-7f31-4373-8ea5-d35d28217538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:51.756113 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc151201-7f31-4373-8ea5-d35d28217538 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:51.756763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc151201-7f31-4373-8ea5-d35d28217538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:51.758365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc151201-7f31-4373-8ea5-d35d28217538 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:51.770010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1029/2033] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:50:51.775136 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258aa126-30b0-420e-8a0b-a10c35638311 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:51.785789 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258aa126-30b0-420e-8a0b-a10c35638311 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:51.790487 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cafc1c-d37b-4607-8c6e-0a049ce9791a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:51.791229 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cafc1c-d37b-4607-8c6e-0a049ce9791a admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:51.791559 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cafc1c-d37b-4607-8c6e-0a049ce9791a admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:51.791832 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cafc1c-d37b-4607-8c6e-0a049ce9791a admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:51.792460 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cafc1c-d37b-4607-8c6e-0a049ce9791a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:51.794500 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cafc1c-d37b-4607-8c6e-0a049ce9791a admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:51.805061 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1005/2034] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:50:51.809266 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc151201-7f31-4373-8ea5-d35d28217538 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:51.820899 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc151201-7f31-4373-8ea5-d35d28217538 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:51.826131 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584042f-99d9-4b32-952f-b286630ce8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:51.826737 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584042f-99d9-4b32-952f-b286630ce8e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:51.827024 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584042f-99d9-4b32-952f-b286630ce8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:51.827303 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584042f-99d9-4b32-952f-b286630ce8e9 admin admin] PATH_INFO: `/v3/projects/e3d8f2d046b34dd197f07c52593b6f60/users/371784d6948c4d2fa496a6cdccb278ec/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:51.854183 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8584042f-99d9-4b32-952f-b286630ce8e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3d8f2d046b34dd197f07c52593b6f60, user_id=371784d6948c4d2fa496a6cdccb278ec, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:51.857536 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8584042f-99d9-4b32-952f-b286630ce8e9 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:51.940343 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1030/2035] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:50:51 2022] PUT /identity/v3/projects/e3d8f2d046b34dd197f07c52593b6f60/users/371784d6948c4d2fa496a6cdccb278ec/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:51.966318 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d05508-f99a-49ea-ade7-be552bf1bb10 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:51.966590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d05508-f99a-49ea-ade7-be552bf1bb10 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:51.967180 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d05508-f99a-49ea-ade7-be552bf1bb10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:52.001522 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-62d05508-f99a-49ea-ade7-be552bf1bb10 None None] MFA Rules not processed for user `371784d6948c4d2fa496a6cdccb278ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:52.002746 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-62d05508-f99a-49ea-ade7-be552bf1bb10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:52.061245 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:52.078529 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1006/2036] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:51 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:52.083279 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584042f-99d9-4b32-952f-b286630ce8e9 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:52.088190 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:52.194382 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584042f-99d9-4b32-952f-b286630ce8e9 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:52.200040 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e10cc6f-db4a-4b43-8731-b5bfcc1feb55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:52.200669 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10cc6f-db4a-4b43-8731-b5bfcc1feb55 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:52.201848 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10cc6f-db4a-4b43-8731-b5bfcc1feb55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:52.201848 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10cc6f-db4a-4b43-8731-b5bfcc1feb55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:52.202162 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e10cc6f-db4a-4b43-8731-b5bfcc1feb55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:52.205528 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e10cc6f-db4a-4b43-8731-b5bfcc1feb55 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:52.290444 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1031/2037] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:50:52 2022] POST /identity/v3/projects => generated 385 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:52.294696 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d05508-f99a-49ea-ade7-be552bf1bb10 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:52.306662 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d05508-f99a-49ea-ade7-be552bf1bb10 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:52.365027 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6daae-815f-427f-9f56-1dcf18af144d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:52.365715 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6daae-815f-427f-9f56-1dcf18af144d admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:52.365999 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6daae-815f-427f-9f56-1dcf18af144d admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:52.366362 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6daae-815f-427f-9f56-1dcf18af144d admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:52.366968 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6daae-815f-427f-9f56-1dcf18af144d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:52.369092 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6daae-815f-427f-9f56-1dcf18af144d admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:52.376079 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e10cc6f-db4a-4b43-8731-b5bfcc1feb55 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:52.378774 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-2e10cc6f-db4a-4b43-8731-b5bfcc1feb55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:52.405894 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e10cc6f-db4a-4b43-8731-b5bfcc1feb55 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:52.413764 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e57775-532a-4a3d-aaa5-17a1dcc4fab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:52.414421 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e57775-532a-4a3d-aaa5-17a1dcc4fab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:52.414704 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e57775-532a-4a3d-aaa5-17a1dcc4fab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:52.414975 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e57775-532a-4a3d-aaa5-17a1dcc4fab5 admin admin] PATH_INFO: `/v3/projects/fdbd476d74bc4d1aad15ef922d9cbd54` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:52.416159 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e57775-532a-4a3d-aaa5-17a1dcc4fab5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdbd476d74bc4d1aad15ef922d9cbd54)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:52.418528 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e57775-532a-4a3d-aaa5-17a1dcc4fab5 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:52.451400 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1007/2038] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:50:52 2022] POST /identity/v3/users => generated 293 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:52.453238 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6daae-815f-427f-9f56-1dcf18af144d admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:52.467382 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6daae-815f-427f-9f56-1dcf18af144d admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:52.480251 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def9353-b5b3-4f60-b720-3b703b1201d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:52.480904 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def9353-b5b3-4f60-b720-3b703b1201d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:52.481186 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def9353-b5b3-4f60-b720-3b703b1201d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:52.481463 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def9353-b5b3-4f60-b720-3b703b1201d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:52.482384 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def9353-b5b3-4f60-b720-3b703b1201d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:52.484245 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def9353-b5b3-4f60-b720-3b703b1201d7 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:52.496485 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1008/2039] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:50:52.500585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def9353-b5b3-4f60-b720-3b703b1201d7 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:52.514142 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def9353-b5b3-4f60-b720-3b703b1201d7 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:52.523156 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de27ac1e-2351-40f0-a997-ec01ae211a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:52.524284 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27ac1e-2351-40f0-a997-ec01ae211a40 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:52.524568 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27ac1e-2351-40f0-a997-ec01ae211a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:52.524846 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27ac1e-2351-40f0-a997-ec01ae211a40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:52.525501 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de27ac1e-2351-40f0-a997-ec01ae211a40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:52.536250 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de27ac1e-2351-40f0-a997-ec01ae211a40 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:52.549325 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1009/2040] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:50:52.552967 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949c4ba-5944-48c8-b82b-bc4760bc5449 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:52.553260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949c4ba-5944-48c8-b82b-bc4760bc5449 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:52.553540 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949c4ba-5944-48c8-b82b-bc4760bc5449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:52.588791 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-2949c4ba-5944-48c8-b82b-bc4760bc5449 None None] MFA Rules not processed for user `78b322344df24e8ca513c19361b79257`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:52.590200 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-2949c4ba-5944-48c8-b82b-bc4760bc5449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:52.592101 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-b5e57775-532a-4a3d-aaa5-17a1dcc4fab5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdbd476d74bc4d1aad15ef922d9cbd54', 'request_id': 'req-b5e57775-532a-4a3d-aaa5-17a1dcc4fab5'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:52.601486 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-b5e57775-532a-4a3d-aaa5-17a1dcc4fab5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdbd476d74bc4d1aad15ef922d9cbd54', 'request_id': 'req-b5e57775-532a-4a3d-aaa5-17a1dcc4fab5'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:52.649770 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:52.669598 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1010/2041] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:52.670475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2949c4ba-5944-48c8-b82b-bc4760bc5449 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:52.693879 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-b5e57775-532a-4a3d-aaa5-17a1dcc4fab5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdbd476d74bc4d1aad15ef922d9cbd54', 'request_id': 'req-b5e57775-532a-4a3d-aaa5-17a1dcc4fab5'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:52.706359 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-b5e57775-532a-4a3d-aaa5-17a1dcc4fab5 admin admin] The token cache is being invalidate because project fdbd476d74bc4d1aad15ef922d9cbd54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:50:52.707221 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-b5e57775-532a-4a3d-aaa5-17a1dcc4fab5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:52.708007 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-b5e57775-532a-4a3d-aaa5-17a1dcc4fab5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:52.709822 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1032/2042] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:50:52 2022] DELETE /identity/v3/projects/fdbd476d74bc4d1aad15ef922d9cbd54 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:52.710498 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e57775-532a-4a3d-aaa5-17a1dcc4fab5 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:52.711878 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-b5e57775-532a-4a3d-aaa5-17a1dcc4fab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:52.736122 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2949c4ba-5944-48c8-b82b-bc4760bc5449 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:52.741379 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0a6a5-9033-400b-b3e6-ba7ff350733c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:52.742045 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0a6a5-9033-400b-b3e6-ba7ff350733c admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:52.742321 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0a6a5-9033-400b-b3e6-ba7ff350733c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:52.742593 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0a6a5-9033-400b-b3e6-ba7ff350733c admin admin] PATH_INFO: `/v3/projects/326a5aacdce14be8aa055816e947c729/users/685fa22a04c3440e8934c70698c48717/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:52.758299 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0a6a5-9033-400b-b3e6-ba7ff350733c admin admin] RBAC: Authorizing `identity:create_grant(project_id=326a5aacdce14be8aa055816e947c729, user_id=685fa22a04c3440e8934c70698c48717, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:52.761042 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0a6a5-9033-400b-b3e6-ba7ff350733c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:52.771063 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e57775-532a-4a3d-aaa5-17a1dcc4fab5 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:52.774845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb374cf-52ad-4088-be5a-744e235dc30d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:52.775486 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb374cf-52ad-4088-be5a-744e235dc30d service cinder] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:52.775767 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb374cf-52ad-4088-be5a-744e235dc30d service cinder] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:52.776037 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb374cf-52ad-4088-be5a-744e235dc30d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:52.777740 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-5cb374cf-52ad-4088-be5a-744e235dc30d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:52.789943 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb374cf-52ad-4088-be5a-744e235dc30d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:52.792092 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb374cf-52ad-4088-be5a-744e235dc30d service cinder] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:52.801940 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1033/2043] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:50:52 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:50:52.802653 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb374cf-52ad-4088-be5a-744e235dc30d service cinder] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:52.803988 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-5cb374cf-52ad-4088-be5a-744e235dc30d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:52.808711 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1011/2044] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:50:52 2022] PUT /identity/v3/projects/326a5aacdce14be8aa055816e947c729/users/685fa22a04c3440e8934c70698c48717/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:52.818583 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb374cf-52ad-4088-be5a-744e235dc30d service cinder] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:52.820696 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2c548-db1d-478a-b886-c6b86fd742f4 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:52.821041 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2c548-db1d-478a-b886-c6b86fd742f4 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:52.821363 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2c548-db1d-478a-b886-c6b86fd742f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:52.828366 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:52.853800 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-a7b2c548-db1d-478a-b886-c6b86fd742f4 None None] MFA Rules not processed for user `685fa22a04c3440e8934c70698c48717`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:52.854724 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a7b2c548-db1d-478a-b886-c6b86fd742f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:52.905369 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e8040-f2e9-4bfc-8ada-35115ad223f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:52.906416 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e8040-f2e9-4bfc-8ada-35115ad223f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:52.907078 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e8040-f2e9-4bfc-8ada-35115ad223f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:52.908221 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e8040-f2e9-4bfc-8ada-35115ad223f7 admin admin] PATH_INFO: `/v3/users/4b1fcba0717f4730a92267a91f9ee4c0` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:52.910340 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e8040-f2e9-4bfc-8ada-35115ad223f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b1fcba0717f4730a92267a91f9ee4c0)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:52.912127 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e8040-f2e9-4bfc-8ada-35115ad223f7 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:52.969905 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1012/2045] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:50:52.975518 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2c548-db1d-478a-b886-c6b86fd742f4 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:52.976977 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a7b2c548-db1d-478a-b886-c6b86fd742f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:52.991068 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2c548-db1d-478a-b886-c6b86fd742f4 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:52.995667 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affef90f-b40f-4461-a82b-cb41d5a9eacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:52.996293 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affef90f-b40f-4461-a82b-cb41d5a9eacb admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:52.996603 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affef90f-b40f-4461-a82b-cb41d5a9eacb admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:52.996876 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affef90f-b40f-4461-a82b-cb41d5a9eacb admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:52.997500 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affef90f-b40f-4461-a82b-cb41d5a9eacb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:52.999420 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affef90f-b40f-4461-a82b-cb41d5a9eacb admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:53.099605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-597e8040-f2e9-4bfc-8ada-35115ad223f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b1fcba0717f4730a92267a91f9ee4c0'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:53.114763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-597e8040-f2e9-4bfc-8ada-35115ad223f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b1fcba0717f4730a92267a91f9ee4c0'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:53.128207 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1013/2046] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:50:52 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:50:53.128207 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affef90f-b40f-4461-a82b-cb41d5a9eacb admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:53.135918 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-597e8040-f2e9-4bfc-8ada-35115ad223f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b1fcba0717f4730a92267a91f9ee4c0'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:53.136101 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-597e8040-f2e9-4bfc-8ada-35115ad223f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b1fcba0717f4730a92267a91f9ee4c0'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:53.184386 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:53.202460 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-597e8040-f2e9-4bfc-8ada-35115ad223f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b1fcba0717f4730a92267a91f9ee4c0'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:53.209272 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1034/2047] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:50:52 2022] DELETE /identity/v3/users/4b1fcba0717f4730a92267a91f9ee4c0 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:53.220296 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affef90f-b40f-4461-a82b-cb41d5a9eacb admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:53.313433 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01131ead-007b-4c96-abb5-871490e21b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:53.314083 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01131ead-007b-4c96-abb5-871490e21b11 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:53.314513 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01131ead-007b-4c96-abb5-871490e21b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:53.314773 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01131ead-007b-4c96-abb5-871490e21b11 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:53.315443 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01131ead-007b-4c96-abb5-871490e21b11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:53.318369 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01131ead-007b-4c96-abb5-871490e21b11 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:53.483851 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1014/2048] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:50:53 2022] POST /identity/v3/users => generated 299 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:50:53.485241 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e8040-f2e9-4bfc-8ada-35115ad223f7 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:53.487201 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-597e8040-f2e9-4bfc-8ada-35115ad223f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:53.509305 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e8040-f2e9-4bfc-8ada-35115ad223f7 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:53.510644 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cbb70b-8065-49a4-9e18-5e6434502e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:53.511164 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cbb70b-8065-49a4-9e18-5e6434502e6f admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:53.511554 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cbb70b-8065-49a4-9e18-5e6434502e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:53.512200 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cbb70b-8065-49a4-9e18-5e6434502e6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:53.517103 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cbb70b-8065-49a4-9e18-5e6434502e6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:53.517103 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cbb70b-8065-49a4-9e18-5e6434502e6f admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:53.527389 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1035/2049] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:50:53.531563 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01131ead-007b-4c96-abb5-871490e21b11 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:53.543282 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01131ead-007b-4c96-abb5-871490e21b11 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:53.549110 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3710f0-2132-4844-b4d0-744015910c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:53.551728 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3710f0-2132-4844-b4d0-744015910c10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:53.551728 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3710f0-2132-4844-b4d0-744015910c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:53.551728 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3710f0-2132-4844-b4d0-744015910c10 admin admin] PATH_INFO: `/v3/projects/62a7ed828e1b467ba65da373e1e97331/users/c71f14c86d5d47fbbdac63e857b782f2/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:53.569667 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3710f0-2132-4844-b4d0-744015910c10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62a7ed828e1b467ba65da373e1e97331, user_id=c71f14c86d5d47fbbdac63e857b782f2, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:53.572426 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3710f0-2132-4844-b4d0-744015910c10 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:53.685801 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cbb70b-8065-49a4-9e18-5e6434502e6f admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:53.687930 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-e3cbb70b-8065-49a4-9e18-5e6434502e6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:53.702490 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cbb70b-8065-49a4-9e18-5e6434502e6f admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:53.707992 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49feda6-ecb9-425c-91d9-582c4fb2d8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:53.708632 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49feda6-ecb9-425c-91d9-582c4fb2d8e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:53.709028 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49feda6-ecb9-425c-91d9-582c4fb2d8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:53.709831 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49feda6-ecb9-425c-91d9-582c4fb2d8e3 admin admin] PATH_INFO: `/v3/projects/3fe8f88ee77e472ba1d75108c5c12aa9` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:53.710983 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49feda6-ecb9-425c-91d9-582c4fb2d8e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fe8f88ee77e472ba1d75108c5c12aa9)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:53.712911 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49feda6-ecb9-425c-91d9-582c4fb2d8e3 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:53.730480 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1015/2050] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:50:53 2022] PUT /identity/v3/projects/62a7ed828e1b467ba65da373e1e97331/users/c71f14c86d5d47fbbdac63e857b782f2/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:50:53.734358 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3710f0-2132-4844-b4d0-744015910c10 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:53.807196 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3710f0-2132-4844-b4d0-744015910c10 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:53.811923 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540810f8-4c82-43e2-83ae-827b44881dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:53.812600 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540810f8-4c82-43e2-83ae-827b44881dc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:53.812882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540810f8-4c82-43e2-83ae-827b44881dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:53.813155 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540810f8-4c82-43e2-83ae-827b44881dc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:53.813817 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540810f8-4c82-43e2-83ae-827b44881dc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:53.815429 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540810f8-4c82-43e2-83ae-827b44881dc4 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:53.826083 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1016/2051] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:50:53.830162 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540810f8-4c82-43e2-83ae-827b44881dc4 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:53.840553 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540810f8-4c82-43e2-83ae-827b44881dc4 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:53.845305 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e249579-cd99-4e10-b87e-ef6770914135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:53.845945 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e249579-cd99-4e10-b87e-ef6770914135 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:53.846219 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e249579-cd99-4e10-b87e-ef6770914135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:53.846539 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e249579-cd99-4e10-b87e-ef6770914135 admin admin] PATH_INFO: `/v3/projects/62a7ed828e1b467ba65da373e1e97331/users/c71f14c86d5d47fbbdac63e857b782f2/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:53.848603 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e249579-cd99-4e10-b87e-ef6770914135 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62a7ed828e1b467ba65da373e1e97331, user_id=c71f14c86d5d47fbbdac63e857b782f2, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:53.851291 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e249579-cd99-4e10-b87e-ef6770914135 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:53.863147 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1017/2052] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:50:53 2022] PUT /identity/v3/projects/62a7ed828e1b467ba65da373e1e97331/users/c71f14c86d5d47fbbdac63e857b782f2/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:50:53.869967 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-a49feda6-ecb9-425c-91d9-582c4fb2d8e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fe8f88ee77e472ba1d75108c5c12aa9', 'request_id': 'req-a49feda6-ecb9-425c-91d9-582c4fb2d8e3'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:53.874133 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe655b82-78cc-45d3-a987-1edd23e3713a None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:53.874581 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe655b82-78cc-45d3-a987-1edd23e3713a None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:53.874866 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe655b82-78cc-45d3-a987-1edd23e3713a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:53.876919 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-a49feda6-ecb9-425c-91d9-582c4fb2d8e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fe8f88ee77e472ba1d75108c5c12aa9', 'request_id': 'req-a49feda6-ecb9-425c-91d9-582c4fb2d8e3'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:53.905866 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-fe655b82-78cc-45d3-a987-1edd23e3713a None None] MFA Rules not processed for user `c71f14c86d5d47fbbdac63e857b782f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:53.907061 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-fe655b82-78cc-45d3-a987-1edd23e3713a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:53.927042 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:53.945725 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-a49feda6-ecb9-425c-91d9-582c4fb2d8e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fe8f88ee77e472ba1d75108c5c12aa9', 'request_id': 'req-a49feda6-ecb9-425c-91d9-582c4fb2d8e3'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:53.958433 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-a49feda6-ecb9-425c-91d9-582c4fb2d8e3 admin admin] The token cache is being invalidate because project 3fe8f88ee77e472ba1d75108c5c12aa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:50:53.958726 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-a49feda6-ecb9-425c-91d9-582c4fb2d8e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:53.959611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-a49feda6-ecb9-425c-91d9-582c4fb2d8e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:53.961363 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1036/2053] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:50:53 2022] DELETE /identity/v3/projects/3fe8f88ee77e472ba1d75108c5c12aa9 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:53.967079 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ba46e-863c-475e-9bd6-5d50d3ab9bba None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:53.968027 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ba46e-863c-475e-9bd6-5d50d3ab9bba None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:53.968323 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ba46e-863c-475e-9bd6-5d50d3ab9bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:53.977444 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1018/2054] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:53 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:50:53.977707 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:53.989682 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-792ba46e-863c-475e-9bd6-5d50d3ab9bba None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:53.990903 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-792ba46e-863c-475e-9bd6-5d50d3ab9bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:53.996802 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe655b82-78cc-45d3-a987-1edd23e3713a None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:53.999044 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-fe655b82-78cc-45d3-a987-1edd23e3713a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:54.059514 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1037/2055] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:50:54.068722 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf6eee-8a10-49e0-847a-ebcd6f106241 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:54.069012 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf6eee-8a10-49e0-847a-ebcd6f106241 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:54.069347 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf6eee-8a10-49e0-847a-ebcd6f106241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:54.073264 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe655b82-78cc-45d3-a987-1edd23e3713a None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:54.076810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f2b3bc-7dda-484c-b339-4e622d2f90d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:54.079811 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2b3bc-7dda-484c-b339-4e622d2f90d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:54.079811 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2b3bc-7dda-484c-b339-4e622d2f90d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:54.079811 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2b3bc-7dda-484c-b339-4e622d2f90d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:54.087353 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f2b3bc-7dda-484c-b339-4e622d2f90d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:54.088809 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f2b3bc-7dda-484c-b339-4e622d2f90d2 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:54.089171 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-4bcf6eee-8a10-49e0-847a-ebcd6f106241 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:54.090378 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-4bcf6eee-8a10-49e0-847a-ebcd6f106241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:54.098673 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1019/2056] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:50:53 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:50:54.100276 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1038/2057] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:50:54.104202 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f2b3bc-7dda-484c-b339-4e622d2f90d2 service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:54.116642 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f2b3bc-7dda-484c-b339-4e622d2f90d2 service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:54.122243 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf123a1-2ce8-4c17-be03-0a6e08913ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:54.122933 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf123a1-2ce8-4c17-be03-0a6e08913ffc admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:54.123228 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf123a1-2ce8-4c17-be03-0a6e08913ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:54.123560 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf123a1-2ce8-4c17-be03-0a6e08913ffc admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:54.124263 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf123a1-2ce8-4c17-be03-0a6e08913ffc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:50:54.124568 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf123a1-2ce8-4c17-be03-0a6e08913ffc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:54.126171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf123a1-2ce8-4c17-be03-0a6e08913ffc admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:54.139960 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1020/2058] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:50:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:50:54.144625 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf6eee-8a10-49e0-847a-ebcd6f106241 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:54.154208 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf6eee-8a10-49e0-847a-ebcd6f106241 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:54.159058 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0a422-4c5e-43ee-a542-7ef0be8adc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:54.159834 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0a422-4c5e-43ee-a542-7ef0be8adc66 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:54.160387 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0a422-4c5e-43ee-a542-7ef0be8adc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:54.160564 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0a422-4c5e-43ee-a542-7ef0be8adc66 admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:54.161242 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a0a422-4c5e-43ee-a542-7ef0be8adc66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:54.163163 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a0a422-4c5e-43ee-a542-7ef0be8adc66 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:54.262176 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1039/2059] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:50:54 2022] POST /identity/v3/projects => generated 387 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:54.266426 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf123a1-2ce8-4c17-be03-0a6e08913ffc admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:54.276827 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf123a1-2ce8-4c17-be03-0a6e08913ffc admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:54.343642 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f370a40-95f4-458b-bbbe-9afbcbd84249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:54.344490 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f370a40-95f4-458b-bbbe-9afbcbd84249 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:54.344775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f370a40-95f4-458b-bbbe-9afbcbd84249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:54.345048 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f370a40-95f4-458b-bbbe-9afbcbd84249 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:54.345771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f370a40-95f4-458b-bbbe-9afbcbd84249 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:54.348153 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f370a40-95f4-458b-bbbe-9afbcbd84249 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:54.468069 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1021/2060] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:50:54 2022] POST /identity/v3/users => generated 294 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:54.470906 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0a422-4c5e-43ee-a542-7ef0be8adc66 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:54.482285 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0a422-4c5e-43ee-a542-7ef0be8adc66 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:54.489340 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3687315e-9cd9-403d-bf69-ba2f5a6a87c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:54.489953 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687315e-9cd9-403d-bf69-ba2f5a6a87c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:54.490229 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687315e-9cd9-403d-bf69-ba2f5a6a87c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:54.490503 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687315e-9cd9-403d-bf69-ba2f5a6a87c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:54.491144 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3687315e-9cd9-403d-bf69-ba2f5a6a87c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:54.492763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3687315e-9cd9-403d-bf69-ba2f5a6a87c7 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:54.505651 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1040/2061] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 12:50:54.507178 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f370a40-95f4-458b-bbbe-9afbcbd84249 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:54.518661 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f370a40-95f4-458b-bbbe-9afbcbd84249 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:54.523674 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40939ada-b7ae-4436-b57f-3a6192aa3a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:54.526848 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40939ada-b7ae-4436-b57f-3a6192aa3a0f admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:54.526848 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40939ada-b7ae-4436-b57f-3a6192aa3a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:54.526848 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40939ada-b7ae-4436-b57f-3a6192aa3a0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:54.526848 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40939ada-b7ae-4436-b57f-3a6192aa3a0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:54.527229 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40939ada-b7ae-4436-b57f-3a6192aa3a0f admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:54.539496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1022/2062] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:50:54.544128 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3687315e-9cd9-403d-bf69-ba2f5a6a87c7 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:54.555642 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3687315e-9cd9-403d-bf69-ba2f5a6a87c7 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:54.561275 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee87e7c6-c7a8-4ee7-bbf1-5d5e7d0abc07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:54.561275 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee87e7c6-c7a8-4ee7-bbf1-5d5e7d0abc07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:54.561507 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee87e7c6-c7a8-4ee7-bbf1-5d5e7d0abc07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:54.561939 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee87e7c6-c7a8-4ee7-bbf1-5d5e7d0abc07 admin admin] PATH_INFO: `/v3/projects/5b3c4f0cba50429ea80f0f39006cb683/users/b65385b7eafa4df1802edf9142cdc766/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:54.583046 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee87e7c6-c7a8-4ee7-bbf1-5d5e7d0abc07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b3c4f0cba50429ea80f0f39006cb683, user_id=b65385b7eafa4df1802edf9142cdc766, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:54.585745 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee87e7c6-c7a8-4ee7-bbf1-5d5e7d0abc07 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:54.634092 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f7950-5e24-4e12-b598-643df9c04d7b None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:54.634399 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f7950-5e24-4e12-b598-643df9c04d7b None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:54.634650 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f7950-5e24-4e12-b598-643df9c04d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:54.654434 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-fc4f7950-5e24-4e12-b598-643df9c04d7b None None] MFA Rules not processed for user `78b322344df24e8ca513c19361b79257`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:54.655664 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-fc4f7950-5e24-4e12-b598-643df9c04d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:54.676865 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:54.712667 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1023/2063] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:50:54.747669 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1041/2064] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:50:54 2022] PUT /identity/v3/projects/5b3c4f0cba50429ea80f0f39006cb683/users/b65385b7eafa4df1802edf9142cdc766/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:54.760907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3dd906-6462-4051-a862-f378628b659a None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:54.761365 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3dd906-6462-4051-a862-f378628b659a None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:54.761831 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3dd906-6462-4051-a862-f378628b659a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:54.792418 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-2a3dd906-6462-4051-a862-f378628b659a None None] MFA Rules not processed for user `b65385b7eafa4df1802edf9142cdc766`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:54.792869 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-2a3dd906-6462-4051-a862-f378628b659a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:54.876245 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1024/2065] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:54 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:54.886693 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee87e7c6-c7a8-4ee7-bbf1-5d5e7d0abc07 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:54.963248 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee87e7c6-c7a8-4ee7-bbf1-5d5e7d0abc07 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:54.968502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68484298-84b9-48fe-818a-07dbb8cfd470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:54.969195 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68484298-84b9-48fe-818a-07dbb8cfd470 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:54.969483 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68484298-84b9-48fe-818a-07dbb8cfd470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:54.969755 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68484298-84b9-48fe-818a-07dbb8cfd470 admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:54.970461 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68484298-84b9-48fe-818a-07dbb8cfd470 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:54.972769 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68484298-84b9-48fe-818a-07dbb8cfd470 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:55.067098 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1042/2066] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:50:54 2022] POST /identity/v3/projects => generated 383 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:55.071575 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3dd906-6462-4051-a862-f378628b659a None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:55.083241 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3dd906-6462-4051-a862-f378628b659a None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:55.180138 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7f03a6-0b1e-41d3-bf9b-839e223ba4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:55.182849 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f03a6-0b1e-41d3-bf9b-839e223ba4e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:55.182849 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f03a6-0b1e-41d3-bf9b-839e223ba4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:55.182849 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f03a6-0b1e-41d3-bf9b-839e223ba4e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:55.182849 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7f03a6-0b1e-41d3-bf9b-839e223ba4e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:55.184544 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7f03a6-0b1e-41d3-bf9b-839e223ba4e9 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:55.293444 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1025/2067] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:50:55 2022] POST /identity/v3/users => generated 298 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:55.299362 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68484298-84b9-48fe-818a-07dbb8cfd470 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:55.310202 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68484298-84b9-48fe-818a-07dbb8cfd470 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:55.317001 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e52a7b-1955-4402-8981-26bbfff3d251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:55.318006 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e52a7b-1955-4402-8981-26bbfff3d251 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:55.318462 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e52a7b-1955-4402-8981-26bbfff3d251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:55.318869 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e52a7b-1955-4402-8981-26bbfff3d251 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:55.319827 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e52a7b-1955-4402-8981-26bbfff3d251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:55.322145 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e52a7b-1955-4402-8981-26bbfff3d251 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:55.344209 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1043/2068] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:50:55.345180 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7f03a6-0b1e-41d3-bf9b-839e223ba4e9 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:55.357077 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7f03a6-0b1e-41d3-bf9b-839e223ba4e9 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:55.362084 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d2fbdd-96a1-4482-9303-43bff07e41b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:55.362838 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2fbdd-96a1-4482-9303-43bff07e41b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:55.365402 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2fbdd-96a1-4482-9303-43bff07e41b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:55.365402 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2fbdd-96a1-4482-9303-43bff07e41b4 admin admin] PATH_INFO: `/v3/projects/8199652728704ff3a247a7c8b58b7ff8/users/a2f6cbd096694eed992350da47327ed8/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:55.388013 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d2fbdd-96a1-4482-9303-43bff07e41b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8199652728704ff3a247a7c8b58b7ff8, user_id=a2f6cbd096694eed992350da47327ed8, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:55.390709 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d2fbdd-96a1-4482-9303-43bff07e41b4 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:55.456087 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e52a7b-1955-4402-8981-26bbfff3d251 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:55.462361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-f8e52a7b-1955-4402-8981-26bbfff3d251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:55.477713 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1026/2069] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:50:55 2022] PUT /identity/v3/projects/8199652728704ff3a247a7c8b58b7ff8/users/a2f6cbd096694eed992350da47327ed8/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:55.481689 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d2fbdd-96a1-4482-9303-43bff07e41b4 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:55.522597 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:55.551269 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:55.577375 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e52a7b-1955-4402-8981-26bbfff3d251 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:55.585983 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d2fbdd-96a1-4482-9303-43bff07e41b4 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:55.600176 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b7a426-9d76-4a2b-ae69-a5a97cd1bcfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:55.601246 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7a426-9d76-4a2b-ae69-a5a97cd1bcfe admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:55.601602 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7a426-9d76-4a2b-ae69-a5a97cd1bcfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:55.602017 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7a426-9d76-4a2b-ae69-a5a97cd1bcfe admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:55.602897 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b7a426-9d76-4a2b-ae69-a5a97cd1bcfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:55.605017 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b7a426-9d76-4a2b-ae69-a5a97cd1bcfe admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:55.630621 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1027/2070] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:50:55.632277 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b7a426-9d76-4a2b-ae69-a5a97cd1bcfe admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:55.645188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b7a426-9d76-4a2b-ae69-a5a97cd1bcfe admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:55.649163 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3897c1-daa1-470c-8f0f-44406dfdb582 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:55.649797 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3897c1-daa1-470c-8f0f-44406dfdb582 service cinder] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:55.650931 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3897c1-daa1-470c-8f0f-44406dfdb582 service cinder] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:55.650931 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3897c1-daa1-470c-8f0f-44406dfdb582 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:55.655254 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f0a73-24a4-4037-a64f-578f60505763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:55.655254 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f0a73-24a4-4037-a64f-578f60505763 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:55.655254 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f0a73-24a4-4037-a64f-578f60505763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:55.655254 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f0a73-24a4-4037-a64f-578f60505763 admin admin] PATH_INFO: `/v3/projects/8199652728704ff3a247a7c8b58b7ff8/users/a2f6cbd096694eed992350da47327ed8/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:55.656642 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f0a73-24a4-4037-a64f-578f60505763 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8199652728704ff3a247a7c8b58b7ff8, user_id=a2f6cbd096694eed992350da47327ed8, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:55.658728 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-3f3897c1-daa1-470c-8f0f-44406dfdb582 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:55.661883 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f0a73-24a4-4037-a64f-578f60505763 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:55.688980 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1028/2071] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:50:55 2022] PUT /identity/v3/projects/8199652728704ff3a247a7c8b58b7ff8/users/a2f6cbd096694eed992350da47327ed8/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:50:55.699619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f9a1f-3fdb-4b3e-9b00-75e6e62e2d10 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:55.699863 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f9a1f-3fdb-4b3e-9b00-75e6e62e2d10 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:55.700274 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f9a1f-3fdb-4b3e-9b00-75e6e62e2d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:55.764120 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-c96f9a1f-3fdb-4b3e-9b00-75e6e62e2d10 None None] MFA Rules not processed for user `a2f6cbd096694eed992350da47327ed8`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:55.765830 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-c96f9a1f-3fdb-4b3e-9b00-75e6e62e2d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:55.774282 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3897c1-daa1-470c-8f0f-44406dfdb582 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:55.776557 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3897c1-daa1-470c-8f0f-44406dfdb582 service cinder] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:55.789386 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1044/2072] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:50:55 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:50:55.865190 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1029/2073] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:55 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:55.881768 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3897c1-daa1-470c-8f0f-44406dfdb582 service cinder] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:55.891681 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3897c1-daa1-470c-8f0f-44406dfdb582 service cinder] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:55.954590 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f6b07-dfdf-43d5-9da4-ad58ba1b3180 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:55.955180 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f6b07-dfdf-43d5-9da4-ad58ba1b3180 service cinder] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:55.955540 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f6b07-dfdf-43d5-9da4-ad58ba1b3180 service cinder] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:55.955818 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f6b07-dfdf-43d5-9da4-ad58ba1b3180 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:55.969163 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2f6b07-dfdf-43d5-9da4-ad58ba1b3180 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:55.970611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2f6b07-dfdf-43d5-9da4-ad58ba1b3180 service cinder] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:55.981929 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1045/2074] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:50:55 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:50:56.056729 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f9a1f-3fdb-4b3e-9b00-75e6e62e2d10 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:56.141064 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f9a1f-3fdb-4b3e-9b00-75e6e62e2d10 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:56.146314 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f06eed-4f8a-4b12-80de-36d2e15fa82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:56.147187 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f06eed-4f8a-4b12-80de-36d2e15fa82d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:56.147504 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f06eed-4f8a-4b12-80de-36d2e15fa82d admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:56.148231 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f06eed-4f8a-4b12-80de-36d2e15fa82d admin admin] PATH_INFO: `/v3/users/b65385b7eafa4df1802edf9142cdc766` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:56.149724 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f06eed-4f8a-4b12-80de-36d2e15fa82d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b65385b7eafa4df1802edf9142cdc766)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:56.152150 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f06eed-4f8a-4b12-80de-36d2e15fa82d admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:56.287044 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:56.296358 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d4f06eed-4f8a-4b12-80de-36d2e15fa82d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b65385b7eafa4df1802edf9142cdc766'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:56.314902 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d4f06eed-4f8a-4b12-80de-36d2e15fa82d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b65385b7eafa4df1802edf9142cdc766'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:56.315978 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d4f06eed-4f8a-4b12-80de-36d2e15fa82d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b65385b7eafa4df1802edf9142cdc766'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:56.324564 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d4f06eed-4f8a-4b12-80de-36d2e15fa82d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b65385b7eafa4df1802edf9142cdc766'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:56.447345 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-d4f06eed-4f8a-4b12-80de-36d2e15fa82d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b65385b7eafa4df1802edf9142cdc766'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:56.457668 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1030/2075] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:50:56 2022] DELETE /identity/v3/users/b65385b7eafa4df1802edf9142cdc766 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:56.463351 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f83ab-2d4d-4642-aa35-cf3489c4c504 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:56.463785 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f83ab-2d4d-4642-aa35-cf3489c4c504 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:56.464225 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f83ab-2d4d-4642-aa35-cf3489c4c504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:56.499593 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-d63f83ab-2d4d-4642-aa35-cf3489c4c504 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:56.507632 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-d63f83ab-2d4d-4642-aa35-cf3489c4c504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:56.540740 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:56.597625 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1046/2076] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:56.607728 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f06eed-4f8a-4b12-80de-36d2e15fa82d admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:56.609587 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d4f06eed-4f8a-4b12-80de-36d2e15fa82d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:56.686917 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f06eed-4f8a-4b12-80de-36d2e15fa82d admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:56.691176 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e3f8a-c1c7-4dc2-bd12-ed5ee5b877a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:56.692155 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e3f8a-c1c7-4dc2-bd12-ed5ee5b877a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:56.692533 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e3f8a-c1c7-4dc2-bd12-ed5ee5b877a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:56.692903 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e3f8a-c1c7-4dc2-bd12-ed5ee5b877a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:56.703988 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e3f8a-c1c7-4dc2-bd12-ed5ee5b877a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:56.705731 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e3f8a-c1c7-4dc2-bd12-ed5ee5b877a5 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:56.716945 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1031/2077] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:50:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:50:57.451003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63f83ab-2d4d-4642-aa35-cf3489c4c504 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:57.452804 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-d63f83ab-2d4d-4642-aa35-cf3489c4c504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:57.472481 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63f83ab-2d4d-4642-aa35-cf3489c4c504 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:57.493061 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77460e8-39a6-4f24-b3bf-f18722dbe4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:57.502622 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77460e8-39a6-4f24-b3bf-f18722dbe4dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:57.504303 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77460e8-39a6-4f24-b3bf-f18722dbe4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:57.505119 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77460e8-39a6-4f24-b3bf-f18722dbe4dc admin admin] PATH_INFO: `/v3/projects/5b3c4f0cba50429ea80f0f39006cb683` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:57.508031 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77460e8-39a6-4f24-b3bf-f18722dbe4dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b3c4f0cba50429ea80f0f39006cb683)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:57.511534 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77460e8-39a6-4f24-b3bf-f18722dbe4dc admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:57.700827 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-e77460e8-39a6-4f24-b3bf-f18722dbe4dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b3c4f0cba50429ea80f0f39006cb683', 'request_id': 'req-e77460e8-39a6-4f24-b3bf-f18722dbe4dc'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:57.720430 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-e77460e8-39a6-4f24-b3bf-f18722dbe4dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b3c4f0cba50429ea80f0f39006cb683', 'request_id': 'req-e77460e8-39a6-4f24-b3bf-f18722dbe4dc'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:57.819368 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-e77460e8-39a6-4f24-b3bf-f18722dbe4dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b3c4f0cba50429ea80f0f39006cb683', 'request_id': 'req-e77460e8-39a6-4f24-b3bf-f18722dbe4dc'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:57.833250 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-e77460e8-39a6-4f24-b3bf-f18722dbe4dc admin admin] The token cache is being invalidate because project 5b3c4f0cba50429ea80f0f39006cb683 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:50:57.833826 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-e77460e8-39a6-4f24-b3bf-f18722dbe4dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:57.834914 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-e77460e8-39a6-4f24-b3bf-f18722dbe4dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:57.836479 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1047/2078] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:50:57 2022] DELETE /identity/v3/projects/5b3c4f0cba50429ea80f0f39006cb683 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:57.840427 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e3f8a-c1c7-4dc2-bd12-ed5ee5b877a5 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:57.850502 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e3f8a-c1c7-4dc2-bd12-ed5ee5b877a5 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:57.952875 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f73cc8-08b2-471c-be11-543419508a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:57.953618 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f73cc8-08b2-471c-be11-543419508a95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:57.953900 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f73cc8-08b2-471c-be11-543419508a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:57.954184 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f73cc8-08b2-471c-be11-543419508a95 admin admin] PATH_INFO: `/v3/users/a2f6cbd096694eed992350da47327ed8` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:57.955518 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f73cc8-08b2-471c-be11-543419508a95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2f6cbd096694eed992350da47327ed8)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:57.958219 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f73cc8-08b2-471c-be11-543419508a95 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:58.152089 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c2f73cc8-08b2-471c-be11-543419508a95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2f6cbd096694eed992350da47327ed8'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:58.164461 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:58.169274 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c2f73cc8-08b2-471c-be11-543419508a95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2f6cbd096694eed992350da47327ed8'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:58.170070 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c2f73cc8-08b2-471c-be11-543419508a95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2f6cbd096694eed992350da47327ed8'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:58.177819 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c2f73cc8-08b2-471c-be11-543419508a95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2f6cbd096694eed992350da47327ed8'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:58.245842 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c2f73cc8-08b2-471c-be11-543419508a95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2f6cbd096694eed992350da47327ed8'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:58.254200 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1032/2079] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:50:57 2022] DELETE /identity/v3/users/a2f6cbd096694eed992350da47327ed8 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:58.892537 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77460e8-39a6-4f24-b3bf-f18722dbe4dc admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:58.897241 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-e77460e8-39a6-4f24-b3bf-f18722dbe4dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:58.922295 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77460e8-39a6-4f24-b3bf-f18722dbe4dc admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:59.079244 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372aba6-c60b-4255-9c03-24149d00840c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:59.083007 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372aba6-c60b-4255-9c03-24149d00840c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:59.083342 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372aba6-c60b-4255-9c03-24149d00840c admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:59.083686 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372aba6-c60b-4255-9c03-24149d00840c admin admin] PATH_INFO: `/v3/projects/8199652728704ff3a247a7c8b58b7ff8` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:59.086236 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9372aba6-c60b-4255-9c03-24149d00840c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8199652728704ff3a247a7c8b58b7ff8)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:59.090636 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9372aba6-c60b-4255-9c03-24149d00840c admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:59.260575 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-9372aba6-c60b-4255-9c03-24149d00840c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8199652728704ff3a247a7c8b58b7ff8', 'request_id': 'req-9372aba6-c60b-4255-9c03-24149d00840c'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:59.271993 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-9372aba6-c60b-4255-9c03-24149d00840c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8199652728704ff3a247a7c8b58b7ff8', 'request_id': 'req-9372aba6-c60b-4255-9c03-24149d00840c'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:59.274845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:50:59.340456 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-9372aba6-c60b-4255-9c03-24149d00840c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8199652728704ff3a247a7c8b58b7ff8', 'request_id': 'req-9372aba6-c60b-4255-9c03-24149d00840c'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:59.360907 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-9372aba6-c60b-4255-9c03-24149d00840c admin admin] The token cache is being invalidate because project 8199652728704ff3a247a7c8b58b7ff8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:50:59.361330 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-9372aba6-c60b-4255-9c03-24149d00840c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:59.362396 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-9372aba6-c60b-4255-9c03-24149d00840c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:50:59.372187 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1048/2080] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:50:58 2022] DELETE /identity/v3/projects/8199652728704ff3a247a7c8b58b7ff8 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:50:59.376906 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1b9b9-05ae-46fa-a23d-fdbc714b524c None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:59.377311 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1b9b9-05ae-46fa-a23d-fdbc714b524c None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:59.380267 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1b9b9-05ae-46fa-a23d-fdbc714b524c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:59.405142 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-35e1b9b9-05ae-46fa-a23d-fdbc714b524c None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:59.406883 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-35e1b9b9-05ae-46fa-a23d-fdbc714b524c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:59.524136 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1033/2081] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:59.531529 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccab1cf-6b19-4c0d-ba7f-99c275d4f2a6 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:59.531816 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccab1cf-6b19-4c0d-ba7f-99c275d4f2a6 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:59.532092 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccab1cf-6b19-4c0d-ba7f-99c275d4f2a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:59.562489 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-fccab1cf-6b19-4c0d-ba7f-99c275d4f2a6 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:50:59.563799 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-fccab1cf-6b19-4c0d-ba7f-99c275d4f2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:50:59.582434 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1049/2082] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:50:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:50:59.586958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1b9b9-05ae-46fa-a23d-fdbc714b524c None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:59.598995 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1b9b9-05ae-46fa-a23d-fdbc714b524c None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:59.606357 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea66c6-d5b7-44b1-86a5-32a42dffc479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:59.607140 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea66c6-d5b7-44b1-86a5-32a42dffc479 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:59.607567 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea66c6-d5b7-44b1-86a5-32a42dffc479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:59.607911 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea66c6-d5b7-44b1-86a5-32a42dffc479 admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:59.611382 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ea66c6-d5b7-44b1-86a5-32a42dffc479 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:50:59.616503 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ea66c6-d5b7-44b1-86a5-32a42dffc479 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:59.620813 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ea66c6-d5b7-44b1-86a5-32a42dffc479 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:59.644001 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccab1cf-6b19-4c0d-ba7f-99c275d4f2a6 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:59.650641 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1034/2083] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:50:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:50:59.654988 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccab1cf-6b19-4c0d-ba7f-99c275d4f2a6 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:59.660336 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5779526-e0df-47e6-b339-c68fe3946a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:59.660948 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5779526-e0df-47e6-b339-c68fe3946a00 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:59.661224 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5779526-e0df-47e6-b339-c68fe3946a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:59.665293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5779526-e0df-47e6-b339-c68fe3946a00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:59.665293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5779526-e0df-47e6-b339-c68fe3946a00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:59.665293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5779526-e0df-47e6-b339-c68fe3946a00 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:59.771250 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea66c6-d5b7-44b1-86a5-32a42dffc479 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:50:59.777157 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1050/2084] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:50:59 2022] POST /identity/v3/projects => generated 381 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:50:59.786478 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea66c6-d5b7-44b1-86a5-32a42dffc479 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:50:59.884472 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3aa7d-0382-4d65-bf55-6f5603d6a948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:50:59.885106 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3aa7d-0382-4d65-bf55-6f5603d6a948 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:50:59.885368 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3aa7d-0382-4d65-bf55-6f5603d6a948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:50:59.885797 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3aa7d-0382-4d65-bf55-6f5603d6a948 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:50:59.886848 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d3aa7d-0382-4d65-bf55-6f5603d6a948 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:50:59.888848 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d3aa7d-0382-4d65-bf55-6f5603d6a948 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:50:59.998626 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1035/2085] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:50:59 2022] POST /identity/v3/users => generated 291 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:51:00.002472 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5779526-e0df-47e6-b339-c68fe3946a00 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:00.013034 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5779526-e0df-47e6-b339-c68fe3946a00 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:00.017775 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff593e5-aedc-4d6b-9dbe-e7e352198e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:00.018390 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff593e5-aedc-4d6b-9dbe-e7e352198e95 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:00.018672 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff593e5-aedc-4d6b-9dbe-e7e352198e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:00.018946 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff593e5-aedc-4d6b-9dbe-e7e352198e95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:00.020335 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff593e5-aedc-4d6b-9dbe-e7e352198e95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:00.021987 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff593e5-aedc-4d6b-9dbe-e7e352198e95 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:00.033463 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1051/2086] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:51:00.037885 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3aa7d-0382-4d65-bf55-6f5603d6a948 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:00.049011 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3aa7d-0382-4d65-bf55-6f5603d6a948 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:00.054421 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b019cd3-edcc-4eee-bf13-343e4c346aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:00.055136 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b019cd3-edcc-4eee-bf13-343e4c346aa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:00.055955 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b019cd3-edcc-4eee-bf13-343e4c346aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:00.055955 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b019cd3-edcc-4eee-bf13-343e4c346aa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:00.056515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b019cd3-edcc-4eee-bf13-343e4c346aa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:00.057944 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b019cd3-edcc-4eee-bf13-343e4c346aa4 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:00.076727 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1036/2087] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:51:00.079494 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff593e5-aedc-4d6b-9dbe-e7e352198e95 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:00.090815 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff593e5-aedc-4d6b-9dbe-e7e352198e95 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:00.096306 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c79f46-0f48-4de3-a2dd-9a23092a9137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:00.101495 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c79f46-0f48-4de3-a2dd-9a23092a9137 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:00.101495 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c79f46-0f48-4de3-a2dd-9a23092a9137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:00.101495 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c79f46-0f48-4de3-a2dd-9a23092a9137 admin admin] PATH_INFO: `/v3/projects/7f341daf8ff14ce7b31e7ee96a4cef23/users/b1d436c4bf2f4a6e8ac26381a65f5545/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:00.119488 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c79f46-0f48-4de3-a2dd-9a23092a9137 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f341daf8ff14ce7b31e7ee96a4cef23, user_id=b1d436c4bf2f4a6e8ac26381a65f5545, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:00.122122 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c79f46-0f48-4de3-a2dd-9a23092a9137 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:00.236373 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b019cd3-edcc-4eee-bf13-343e4c346aa4 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:00.239124 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-6b019cd3-edcc-4eee-bf13-343e4c346aa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:00.261873 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b019cd3-edcc-4eee-bf13-343e4c346aa4 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:00.263561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1052/2088] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:51:00 2022] PUT /identity/v3/projects/7f341daf8ff14ce7b31e7ee96a4cef23/users/b1d436c4bf2f4a6e8ac26381a65f5545/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:00.284212 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3c1c9-25d5-4ef1-b70f-f52dea794cdd None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:00.286473 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3c1c9-25d5-4ef1-b70f-f52dea794cdd None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:00.286822 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3c1c9-25d5-4ef1-b70f-f52dea794cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:00.318010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-a8b3c1c9-25d5-4ef1-b70f-f52dea794cdd None None] MFA Rules not processed for user `b1d436c4bf2f4a6e8ac26381a65f5545`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:00.319414 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-a8b3c1c9-25d5-4ef1-b70f-f52dea794cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:00.352162 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:00.376729 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cefc85-c526-40bd-a9f8-672ac3dd0dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:00.377473 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cefc85-c526-40bd-a9f8-672ac3dd0dc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:00.378142 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cefc85-c526-40bd-a9f8-672ac3dd0dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:00.378359 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cefc85-c526-40bd-a9f8-672ac3dd0dc1 admin admin] PATH_INFO: `/v3/users/f938505064304ae19cd0a6df370b7721` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:00.380651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cefc85-c526-40bd-a9f8-672ac3dd0dc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f938505064304ae19cd0a6df370b7721)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:00.382939 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cefc85-c526-40bd-a9f8-672ac3dd0dc1 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:00.411864 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1053/2089] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:00 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:00.423843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b3c1c9-25d5-4ef1-b70f-f52dea794cdd None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:00.435107 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b3c1c9-25d5-4ef1-b70f-f52dea794cdd None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:00.440604 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c441e897-03c1-4271-aa13-0d71c0f1c2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:00.444266 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441e897-03c1-4271-aa13-0d71c0f1c2f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:00.444266 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441e897-03c1-4271-aa13-0d71c0f1c2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:00.444266 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441e897-03c1-4271-aa13-0d71c0f1c2f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:00.444266 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c441e897-03c1-4271-aa13-0d71c0f1c2f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:00.445890 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c441e897-03c1-4271-aa13-0d71c0f1c2f1 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:00.576516 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1054/2090] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:51:00 2022] POST /identity/v3/projects => generated 383 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:51:00.580725 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c441e897-03c1-4271-aa13-0d71c0f1c2f1 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:00.595133 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-44cefc85-c526-40bd-a9f8-672ac3dd0dc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f938505064304ae19cd0a6df370b7721'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:00.606975 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-44cefc85-c526-40bd-a9f8-672ac3dd0dc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f938505064304ae19cd0a6df370b7721'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:00.607719 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-44cefc85-c526-40bd-a9f8-672ac3dd0dc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f938505064304ae19cd0a6df370b7721'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:00.614936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-44cefc85-c526-40bd-a9f8-672ac3dd0dc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f938505064304ae19cd0a6df370b7721'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:00.630635 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:00.646807 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c441e897-03c1-4271-aa13-0d71c0f1c2f1 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:00.656640 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf7ed4-a01d-4e27-a363-f8b1e7a5a160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:00.657286 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf7ed4-a01d-4e27-a363-f8b1e7a5a160 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:00.657568 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf7ed4-a01d-4e27-a363-f8b1e7a5a160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:00.657838 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf7ed4-a01d-4e27-a363-f8b1e7a5a160 admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:00.658572 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcf7ed4-a01d-4e27-a363-f8b1e7a5a160 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:00.660650 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcf7ed4-a01d-4e27-a363-f8b1e7a5a160 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:00.688265 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-44cefc85-c526-40bd-a9f8-672ac3dd0dc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f938505064304ae19cd0a6df370b7721'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:00.695939 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1037/2091] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:51:00 2022] DELETE /identity/v3/users/f938505064304ae19cd0a6df370b7721 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:00.756418 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1055/2092] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:51:00 2022] POST /identity/v3/users => generated 298 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:51:00.761131 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cefc85-c526-40bd-a9f8-672ac3dd0dc1 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:00.763512 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-44cefc85-c526-40bd-a9f8-672ac3dd0dc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:00.857472 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cefc85-c526-40bd-a9f8-672ac3dd0dc1 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:00.865628 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b03b5-56cc-4ab9-8acc-774cd62f0654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:00.866436 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b03b5-56cc-4ab9-8acc-774cd62f0654 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:00.866875 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b03b5-56cc-4ab9-8acc-774cd62f0654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:00.867442 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b03b5-56cc-4ab9-8acc-774cd62f0654 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:00.868388 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347b03b5-56cc-4ab9-8acc-774cd62f0654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:00.870187 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347b03b5-56cc-4ab9-8acc-774cd62f0654 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:00.883777 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1038/2093] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:00 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:51:00.887739 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf7ed4-a01d-4e27-a363-f8b1e7a5a160 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:00.900735 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf7ed4-a01d-4e27-a363-f8b1e7a5a160 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:00.914531 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f63d8d0-a305-4ec6-b9b7-09691c9aaea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:00.920581 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63d8d0-a305-4ec6-b9b7-09691c9aaea4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:00.920581 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63d8d0-a305-4ec6-b9b7-09691c9aaea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:00.920581 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63d8d0-a305-4ec6-b9b7-09691c9aaea4 admin admin] PATH_INFO: `/v3/projects/cf0cbb8172a943ef8ce6e156cbc71eaa/users/c4251514859f4af8bd8fb8baa65499ef/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:00.935278 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f63d8d0-a305-4ec6-b9b7-09691c9aaea4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf0cbb8172a943ef8ce6e156cbc71eaa, user_id=c4251514859f4af8bd8fb8baa65499ef, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:00.938750 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f63d8d0-a305-4ec6-b9b7-09691c9aaea4 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:01.043823 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1056/2094] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:51:00 2022] PUT /identity/v3/projects/cf0cbb8172a943ef8ce6e156cbc71eaa/users/c4251514859f4af8bd8fb8baa65499ef/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:51:01.047745 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b03b5-56cc-4ab9-8acc-774cd62f0654 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:01.065675 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b03b5-56cc-4ab9-8acc-774cd62f0654 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:01.121688 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f63d8d0-a305-4ec6-b9b7-09691c9aaea4 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:01.123512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-0f63d8d0-a305-4ec6-b9b7-09691c9aaea4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:01.148535 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:01.165757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21361708-782e-4a95-8500-d198ebd63805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:01.168184 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21361708-782e-4a95-8500-d198ebd63805 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:01.168500 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21361708-782e-4a95-8500-d198ebd63805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:01.168816 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21361708-782e-4a95-8500-d198ebd63805 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:01.169479 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21361708-782e-4a95-8500-d198ebd63805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:01.171343 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21361708-782e-4a95-8500-d198ebd63805 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:01.183147 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1039/2095] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 12:51:01.187183 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21361708-782e-4a95-8500-d198ebd63805 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:01.197916 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21361708-782e-4a95-8500-d198ebd63805 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:01.206420 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07909010-951e-41a5-bd14-f8fd4a7bf5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:01.213918 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07909010-951e-41a5-bd14-f8fd4a7bf5c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:01.214504 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07909010-951e-41a5-bd14-f8fd4a7bf5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:01.214796 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07909010-951e-41a5-bd14-f8fd4a7bf5c7 admin admin] PATH_INFO: `/v3/projects/cf0cbb8172a943ef8ce6e156cbc71eaa/users/c4251514859f4af8bd8fb8baa65499ef/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:01.217518 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07909010-951e-41a5-bd14-f8fd4a7bf5c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf0cbb8172a943ef8ce6e156cbc71eaa, user_id=c4251514859f4af8bd8fb8baa65499ef, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:01.217678 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f63d8d0-a305-4ec6-b9b7-09691c9aaea4 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:01.220961 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07909010-951e-41a5-bd14-f8fd4a7bf5c7 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:01.223147 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a6b675-95b8-4a4a-8131-89772963ecf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:01.224176 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6b675-95b8-4a4a-8131-89772963ecf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:01.224460 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6b675-95b8-4a4a-8131-89772963ecf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:01.224737 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6b675-95b8-4a4a-8131-89772963ecf2 admin admin] PATH_INFO: `/v3/projects/d7994e24004a42ac88491cf70b3d1086` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:01.225916 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a6b675-95b8-4a4a-8131-89772963ecf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7994e24004a42ac88491cf70b3d1086)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:01.230630 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a6b675-95b8-4a4a-8131-89772963ecf2 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:01.256590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1040/2096] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:51:01 2022] PUT /identity/v3/projects/cf0cbb8172a943ef8ce6e156cbc71eaa/users/c4251514859f4af8bd8fb8baa65499ef/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:51:01.258436 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07909010-951e-41a5-bd14-f8fd4a7bf5c7 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:01.260162 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-07909010-951e-41a5-bd14-f8fd4a7bf5c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:01.333553 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07909010-951e-41a5-bd14-f8fd4a7bf5c7 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:01.338890 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c385d813-057b-4a2f-8768-9b59f76944b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:01.339949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c385d813-057b-4a2f-8768-9b59f76944b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:01.340375 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c385d813-057b-4a2f-8768-9b59f76944b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:01.340807 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c385d813-057b-4a2f-8768-9b59f76944b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:01.343435 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-c385d813-057b-4a2f-8768-9b59f76944b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:01.354885 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-f6a6b675-95b8-4a4a-8131-89772963ecf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7994e24004a42ac88491cf70b3d1086', 'request_id': 'req-f6a6b675-95b8-4a4a-8131-89772963ecf2'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:01.362167 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-f6a6b675-95b8-4a4a-8131-89772963ecf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7994e24004a42ac88491cf70b3d1086', 'request_id': 'req-f6a6b675-95b8-4a4a-8131-89772963ecf2'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:01.419915 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-f6a6b675-95b8-4a4a-8131-89772963ecf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7994e24004a42ac88491cf70b3d1086', 'request_id': 'req-f6a6b675-95b8-4a4a-8131-89772963ecf2'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:01.422589 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c385d813-057b-4a2f-8768-9b59f76944b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:01.424143 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c385d813-057b-4a2f-8768-9b59f76944b5 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:01.435990 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-f6a6b675-95b8-4a4a-8131-89772963ecf2 admin admin] The token cache is being invalidate because project d7994e24004a42ac88491cf70b3d1086 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:51:01.436211 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1041/2097] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:51:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:51:01.437239 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb819e31-7466-4405-8780-3bcb93e982b1 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:01.437533 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb819e31-7466-4405-8780-3bcb93e982b1 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:01.437808 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb819e31-7466-4405-8780-3bcb93e982b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:01.453338 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-f6a6b675-95b8-4a4a-8131-89772963ecf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:01.455279 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-f6a6b675-95b8-4a4a-8131-89772963ecf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:01.459437 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1057/2098] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:51:01 2022] DELETE /identity/v3/projects/d7994e24004a42ac88491cf70b3d1086 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:01.465773 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a3a4f-f66e-414f-823f-715ecc519143 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:01.466003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a3a4f-f66e-414f-823f-715ecc519143 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:01.466296 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a3a4f-f66e-414f-823f-715ecc519143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:01.510769 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-8d1a3a4f-f66e-414f-823f-715ecc519143 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:01.512032 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-8d1a3a4f-f66e-414f-823f-715ecc519143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:01.517987 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-eb819e31-7466-4405-8780-3bcb93e982b1 None None] MFA Rules not processed for user `c4251514859f4af8bd8fb8baa65499ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:01.532988 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-eb819e31-7466-4405-8780-3bcb93e982b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:01.633882 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:01.685391 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1058/2099] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:01.693846 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec979f-376e-4610-9d00-6aa115d458d3 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:01.694219 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec979f-376e-4610-9d00-6aa115d458d3 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:01.694558 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec979f-376e-4610-9d00-6aa115d458d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:01.695073 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1042/2100] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:01 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:01.705895 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb819e31-7466-4405-8780-3bcb93e982b1 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:01.707832 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-eb819e31-7466-4405-8780-3bcb93e982b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:01.719159 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-93ec979f-376e-4610-9d00-6aa115d458d3 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:01.720843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-93ec979f-376e-4610-9d00-6aa115d458d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:01.730653 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:01.733200 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1059/2101] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:51:01.737277 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec979f-376e-4610-9d00-6aa115d458d3 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:01.747029 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec979f-376e-4610-9d00-6aa115d458d3 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:01.759017 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ef613-26d0-4541-a8e4-a848e3cd439b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:01.759347 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ef613-26d0-4541-a8e4-a848e3cd439b admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:01.759684 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ef613-26d0-4541-a8e4-a848e3cd439b admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:01.759931 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ef613-26d0-4541-a8e4-a848e3cd439b admin admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:01.760735 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ef613-26d0-4541-a8e4-a848e3cd439b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:51:01.761055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ef613-26d0-4541-a8e4-a848e3cd439b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:01.762669 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ef613-26d0-4541-a8e4-a848e3cd439b admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:01.782686 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1060/2102] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:51:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:51:01.782920 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb819e31-7466-4405-8780-3bcb93e982b1 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:01.784568 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ef613-26d0-4541-a8e4-a848e3cd439b admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:01.784729 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75982eb-f838-4860-9d7c-4428bd70a497 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:01.786117 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75982eb-f838-4860-9d7c-4428bd70a497 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:01.786404 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75982eb-f838-4860-9d7c-4428bd70a497 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:01.786680 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75982eb-f838-4860-9d7c-4428bd70a497 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:01.796277 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ef613-26d0-4541-a8e4-a848e3cd439b admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:01.797105 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75982eb-f838-4860-9d7c-4428bd70a497 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:01.798453 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75982eb-f838-4860-9d7c-4428bd70a497 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:01.802301 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583028ba-4d6c-4a2c-9514-48f7deca7b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:01.802929 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583028ba-4d6c-4a2c-9514-48f7deca7b46 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:01.803364 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583028ba-4d6c-4a2c-9514-48f7deca7b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:01.803643 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583028ba-4d6c-4a2c-9514-48f7deca7b46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:01.804285 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583028ba-4d6c-4a2c-9514-48f7deca7b46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:01.806178 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583028ba-4d6c-4a2c-9514-48f7deca7b46 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:01.809826 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1043/2103] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:51:01 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:51:01.880942 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1061/2104] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:51:01 2022] POST /identity/v3/projects => generated 363 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:51:01.885296 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75982eb-f838-4860-9d7c-4428bd70a497 service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:01.898269 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75982eb-f838-4860-9d7c-4428bd70a497 service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:01.953849 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583028ba-4d6c-4a2c-9514-48f7deca7b46 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:01.998524 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c4de3-0e3e-47c3-9eef-b2a3e497912d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:01.999419 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c4de3-0e3e-47c3-9eef-b2a3e497912d admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:01.999985 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c4de3-0e3e-47c3-9eef-b2a3e497912d admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:02.000373 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c4de3-0e3e-47c3-9eef-b2a3e497912d admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:02.001017 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042c4de3-0e3e-47c3-9eef-b2a3e497912d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:02.003111 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042c4de3-0e3e-47c3-9eef-b2a3e497912d admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:02.029008 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583028ba-4d6c-4a2c-9514-48f7deca7b46 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:02.036268 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5d37d-52ad-4e76-beba-33c7e8ca87df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:02.037441 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5d37d-52ad-4e76-beba-33c7e8ca87df service cinder] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:02.037842 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5d37d-52ad-4e76-beba-33c7e8ca87df service cinder] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:02.038120 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5d37d-52ad-4e76-beba-33c7e8ca87df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:02.041151 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-a5f5d37d-52ad-4e76-beba-33c7e8ca87df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:02.074373 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1044/2105] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:51:01 2022] POST /identity/v3/users => generated 282 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:51:02.078299 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c4de3-0e3e-47c3-9eef-b2a3e497912d admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:02.096610 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c4de3-0e3e-47c3-9eef-b2a3e497912d admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:02.101986 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18470d20-2dfd-4328-93c6-b86f565c6f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:02.102684 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18470d20-2dfd-4328-93c6-b86f565c6f60 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:02.103031 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18470d20-2dfd-4328-93c6-b86f565c6f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:02.103419 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18470d20-2dfd-4328-93c6-b86f565c6f60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:02.104203 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18470d20-2dfd-4328-93c6-b86f565c6f60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:02.106052 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18470d20-2dfd-4328-93c6-b86f565c6f60 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:02.110282 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f5d37d-52ad-4e76-beba-33c7e8ca87df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:02.112176 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f5d37d-52ad-4e76-beba-33c7e8ca87df service cinder] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:02.118766 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1045/2106] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:51:02.122818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18470d20-2dfd-4328-93c6-b86f565c6f60 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:02.129377 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1062/2107] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:51:01 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:51:02.136831 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18470d20-2dfd-4328-93c6-b86f565c6f60 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:02.142742 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69ad4d-1525-4d0b-9831-dcb1be4e5464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:02.143642 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69ad4d-1525-4d0b-9831-dcb1be4e5464 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:02.143993 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69ad4d-1525-4d0b-9831-dcb1be4e5464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:02.145073 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69ad4d-1525-4d0b-9831-dcb1be4e5464 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:02.145718 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e69ad4d-1525-4d0b-9831-dcb1be4e5464 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:02.148454 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e69ad4d-1525-4d0b-9831-dcb1be4e5464 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:02.161776 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1046/2108] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:51:02.166688 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5d37d-52ad-4e76-beba-33c7e8ca87df service cinder] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:02.192259 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5d37d-52ad-4e76-beba-33c7e8ca87df service cinder] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:02.204718 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b769ab-7ce8-4af8-bcf8-0c12e00f41be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:02.205420 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b769ab-7ce8-4af8-bcf8-0c12e00f41be admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:02.205730 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b769ab-7ce8-4af8-bcf8-0c12e00f41be admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:02.206017 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b769ab-7ce8-4af8-bcf8-0c12e00f41be admin admin] PATH_INFO: `/v3/projects/ad6748e5b9094b4c8ef92209e458abaf/users/c7a7804e81be4e76bbbe4ecc223f423d/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:02.224442 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b769ab-7ce8-4af8-bcf8-0c12e00f41be admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad6748e5b9094b4c8ef92209e458abaf, user_id=c7a7804e81be4e76bbbe4ecc223f423d, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:02.227117 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b769ab-7ce8-4af8-bcf8-0c12e00f41be admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:02.340452 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1063/2109] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:51:02 2022] PUT /identity/v3/projects/ad6748e5b9094b4c8ef92209e458abaf/users/c7a7804e81be4e76bbbe4ecc223f423d/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:02.351399 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961d89b-fee6-4725-9835-361de1abf33e None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:02.351659 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961d89b-fee6-4725-9835-361de1abf33e None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:02.351934 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961d89b-fee6-4725-9835-361de1abf33e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:02.383106 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-d961d89b-fee6-4725-9835-361de1abf33e None None] MFA Rules not processed for user `c7a7804e81be4e76bbbe4ecc223f423d`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:02.384381 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d961d89b-fee6-4725-9835-361de1abf33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:02.454637 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1047/2110] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:02.458803 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b769ab-7ce8-4af8-bcf8-0c12e00f41be admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:02.495864 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:02.529548 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b769ab-7ce8-4af8-bcf8-0c12e00f41be admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:02.534406 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e064f-9b78-473a-98dd-6b44e7deccf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:02.535243 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e064f-9b78-473a-98dd-6b44e7deccf4 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:02.535708 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e064f-9b78-473a-98dd-6b44e7deccf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:02.535987 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e064f-9b78-473a-98dd-6b44e7deccf4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:02.536626 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e064f-9b78-473a-98dd-6b44e7deccf4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:02.538533 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e064f-9b78-473a-98dd-6b44e7deccf4 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:02.661938 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1064/2111] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:51:02 2022] POST /identity/v3/projects => generated 363 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:51:02.666346 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d961d89b-fee6-4725-9835-361de1abf33e None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:02.678112 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d961d89b-fee6-4725-9835-361de1abf33e None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:02.700883 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:02.753208 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906bdc0-41e5-4d81-a285-b5ae44de992a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:02.753833 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906bdc0-41e5-4d81-a285-b5ae44de992a admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:02.754180 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906bdc0-41e5-4d81-a285-b5ae44de992a admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:02.754444 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906bdc0-41e5-4d81-a285-b5ae44de992a admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:02.755353 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a906bdc0-41e5-4d81-a285-b5ae44de992a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:02.757276 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a906bdc0-41e5-4d81-a285-b5ae44de992a admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:02.850664 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1048/2112] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:51:02 2022] POST /identity/v3/users => generated 288 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:51:02.854677 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e064f-9b78-473a-98dd-6b44e7deccf4 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:02.877709 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e064f-9b78-473a-98dd-6b44e7deccf4 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:02.886568 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a7bc5d-a93a-4cb1-a829-7738378637d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:02.887186 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7bc5d-a93a-4cb1-a829-7738378637d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:02.887549 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7bc5d-a93a-4cb1-a829-7738378637d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:02.887816 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7bc5d-a93a-4cb1-a829-7738378637d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:02.888455 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a7bc5d-a93a-4cb1-a829-7738378637d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:02.890013 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a7bc5d-a93a-4cb1-a829-7738378637d2 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:02.903823 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1065/2113] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:02 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:51:02.908591 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906bdc0-41e5-4d81-a285-b5ae44de992a admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:02.922079 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906bdc0-41e5-4d81-a285-b5ae44de992a admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:02.928184 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b76889e-5673-4a3a-abbd-f72c18e6a40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:02.928934 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76889e-5673-4a3a-abbd-f72c18e6a40a admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:02.929273 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76889e-5673-4a3a-abbd-f72c18e6a40a admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:02.929611 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76889e-5673-4a3a-abbd-f72c18e6a40a admin admin] PATH_INFO: `/v3/projects/99c124db1b694dc790e9a0062fa72d26/users/e80af2d74ff9428e841b95352fc16f9e/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:02.948160 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b76889e-5673-4a3a-abbd-f72c18e6a40a admin admin] RBAC: Authorizing `identity:create_grant(project_id=99c124db1b694dc790e9a0062fa72d26, user_id=e80af2d74ff9428e841b95352fc16f9e, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:02.950917 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b76889e-5673-4a3a-abbd-f72c18e6a40a admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:03.021142 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1049/2114] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:51:02 2022] PUT /identity/v3/projects/99c124db1b694dc790e9a0062fa72d26/users/e80af2d74ff9428e841b95352fc16f9e/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:03.025031 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a7bc5d-a93a-4cb1-a829-7738378637d2 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:03.035402 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a7bc5d-a93a-4cb1-a829-7738378637d2 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:03.095443 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba361869-6440-4aa5-a9b9-0077677d22cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:03.096053 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba361869-6440-4aa5-a9b9-0077677d22cd admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:03.096336 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba361869-6440-4aa5-a9b9-0077677d22cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:03.096612 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba361869-6440-4aa5-a9b9-0077677d22cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:03.097863 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba361869-6440-4aa5-a9b9-0077677d22cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:03.099544 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba361869-6440-4aa5-a9b9-0077677d22cd admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:03.109549 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1066/2115] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:03 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 12:51:03.114370 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b76889e-5673-4a3a-abbd-f72c18e6a40a admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:03.125805 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b76889e-5673-4a3a-abbd-f72c18e6a40a admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:03.131193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1030307-dbac-40a2-8206-5bb3c9d8dc0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:03.131896 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1030307-dbac-40a2-8206-5bb3c9d8dc0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:03.132176 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1030307-dbac-40a2-8206-5bb3c9d8dc0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:03.132448 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1030307-dbac-40a2-8206-5bb3c9d8dc0b admin admin] PATH_INFO: `/v3/projects/99c124db1b694dc790e9a0062fa72d26/users/e80af2d74ff9428e841b95352fc16f9e/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:03.134349 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1030307-dbac-40a2-8206-5bb3c9d8dc0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=99c124db1b694dc790e9a0062fa72d26, user_id=e80af2d74ff9428e841b95352fc16f9e, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:03.137019 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1030307-dbac-40a2-8206-5bb3c9d8dc0b admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:03.148628 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1050/2116] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:51:03 2022] PUT /identity/v3/projects/99c124db1b694dc790e9a0062fa72d26/users/e80af2d74ff9428e841b95352fc16f9e/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:51:03.161306 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ee502-8bf6-47b6-b43b-c02786014782 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:03.161819 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ee502-8bf6-47b6-b43b-c02786014782 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:03.161912 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ee502-8bf6-47b6-b43b-c02786014782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:03.190976 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-992ee502-8bf6-47b6-b43b-c02786014782 None None] MFA Rules not processed for user `e80af2d74ff9428e841b95352fc16f9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:03.192373 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-992ee502-8bf6-47b6-b43b-c02786014782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:03.284181 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1067/2117] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:03 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:03.296578 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1030307-dbac-40a2-8206-5bb3c9d8dc0b admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:03.365824 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1030307-dbac-40a2-8206-5bb3c9d8dc0b admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:03.369669 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e3cca-c1bd-47a9-bf77-1ccdb4d450c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:03.370435 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e3cca-c1bd-47a9-bf77-1ccdb4d450c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:03.370717 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e3cca-c1bd-47a9-bf77-1ccdb4d450c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:03.370989 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e3cca-c1bd-47a9-bf77-1ccdb4d450c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:03.385374 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3e3cca-c1bd-47a9-bf77-1ccdb4d450c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:03.385374 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3e3cca-c1bd-47a9-bf77-1ccdb4d450c8 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:03.393180 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1051/2118] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:51:03 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:51:03.668942 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ee502-8bf6-47b6-b43b-c02786014782 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:03.670330 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-992ee502-8bf6-47b6-b43b-c02786014782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:03.682527 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ee502-8bf6-47b6-b43b-c02786014782 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:03.686171 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d8358-85f7-448c-8d4b-b6ad0f2085da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:03.686775 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d8358-85f7-448c-8d4b-b6ad0f2085da service cinder] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:03.687053 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d8358-85f7-448c-8d4b-b6ad0f2085da service cinder] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:03.687342 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d8358-85f7-448c-8d4b-b6ad0f2085da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:03.712521 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:03.747042 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d8358-85f7-448c-8d4b-b6ad0f2085da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:03.748607 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d8358-85f7-448c-8d4b-b6ad0f2085da service cinder] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:03.757464 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1068/2119] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:51:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:51:05.431656 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e3cca-c1bd-47a9-bf77-1ccdb4d450c8 service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:05.437222 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-3e3e3cca-c1bd-47a9-bf77-1ccdb4d450c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:05.501012 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e3cca-c1bd-47a9-bf77-1ccdb4d450c8 service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:05.506289 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee793235-c5a8-4981-838d-4335faa45fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:05.506953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee793235-c5a8-4981-838d-4335faa45fa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:05.507235 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee793235-c5a8-4981-838d-4335faa45fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:05.507542 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee793235-c5a8-4981-838d-4335faa45fa7 admin admin] PATH_INFO: `/v3/users/b1d436c4bf2f4a6e8ac26381a65f5545` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:05.508661 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee793235-c5a8-4981-838d-4335faa45fa7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1d436c4bf2f4a6e8ac26381a65f5545)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:05.511700 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee793235-c5a8-4981-838d-4335faa45fa7 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:05.527183 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:05.836480 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ee793235-c5a8-4981-838d-4335faa45fa7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1d436c4bf2f4a6e8ac26381a65f5545'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:05.850754 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ee793235-c5a8-4981-838d-4335faa45fa7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1d436c4bf2f4a6e8ac26381a65f5545'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:05.851548 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ee793235-c5a8-4981-838d-4335faa45fa7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1d436c4bf2f4a6e8ac26381a65f5545'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:05.858444 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ee793235-c5a8-4981-838d-4335faa45fa7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1d436c4bf2f4a6e8ac26381a65f5545'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:05.917046 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ee793235-c5a8-4981-838d-4335faa45fa7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1d436c4bf2f4a6e8ac26381a65f5545'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:05.925363 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1052/2120] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:51:05 2022] DELETE /identity/v3/users/b1d436c4bf2f4a6e8ac26381a65f5545 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:05.940590 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292e58b-d73f-4740-8bcd-be9792385cc8 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:05.940590 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292e58b-d73f-4740-8bcd-be9792385cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:05.940590 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292e58b-d73f-4740-8bcd-be9792385cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:05.951923 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-4292e58b-d73f-4740-8bcd-be9792385cc8 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:05.953125 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-4292e58b-d73f-4740-8bcd-be9792385cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:06.018074 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee793235-c5a8-4981-838d-4335faa45fa7 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:06.021626 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ee793235-c5a8-4981-838d-4335faa45fa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:06.027228 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1069/2121] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:06.044497 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292e58b-d73f-4740-8bcd-be9792385cc8 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:06.048639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-4292e58b-d73f-4740-8bcd-be9792385cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:06.051192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee793235-c5a8-4981-838d-4335faa45fa7 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:06.057513 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97355ac-9604-4a8e-8f87-2bf448d685b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:06.058323 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97355ac-9604-4a8e-8f87-2bf448d685b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:06.058609 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97355ac-9604-4a8e-8f87-2bf448d685b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:06.059091 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97355ac-9604-4a8e-8f87-2bf448d685b1 admin admin] PATH_INFO: `/v3/users/371784d6948c4d2fa496a6cdccb278ec` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:06.060435 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97355ac-9604-4a8e-8f87-2bf448d685b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=371784d6948c4d2fa496a6cdccb278ec)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:06.062901 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97355ac-9604-4a8e-8f87-2bf448d685b1 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:06.115478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292e58b-d73f-4740-8bcd-be9792385cc8 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:06.119095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b806e-c880-44c1-b564-1df2377125ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:06.119735 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b806e-c880-44c1-b564-1df2377125ab service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:06.120012 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b806e-c880-44c1-b564-1df2377125ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:06.120283 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b806e-c880-44c1-b564-1df2377125ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:06.129933 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8b806e-c880-44c1-b564-1df2377125ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:06.131370 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8b806e-c880-44c1-b564-1df2377125ab service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:06.140800 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1070/2122] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:51:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:51:06.278534 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e97355ac-9604-4a8e-8f87-2bf448d685b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '371784d6948c4d2fa496a6cdccb278ec'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:06.303997 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e97355ac-9604-4a8e-8f87-2bf448d685b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '371784d6948c4d2fa496a6cdccb278ec'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:06.304735 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e97355ac-9604-4a8e-8f87-2bf448d685b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '371784d6948c4d2fa496a6cdccb278ec'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:06.313011 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e97355ac-9604-4a8e-8f87-2bf448d685b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '371784d6948c4d2fa496a6cdccb278ec'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:06.421691 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e97355ac-9604-4a8e-8f87-2bf448d685b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '371784d6948c4d2fa496a6cdccb278ec'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:06.435722 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1053/2123] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:51:06 2022] DELETE /identity/v3/users/371784d6948c4d2fa496a6cdccb278ec => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:06.440476 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211784cf-af70-4503-8164-195583804d98 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:06.440785 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211784cf-af70-4503-8164-195583804d98 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:06.441115 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211784cf-af70-4503-8164-195583804d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:06.463340 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-211784cf-af70-4503-8164-195583804d98 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:06.465245 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-211784cf-af70-4503-8164-195583804d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:06.533750 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1071/2124] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:06.544277 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97355ac-9604-4a8e-8f87-2bf448d685b1 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:06.545694 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e97355ac-9604-4a8e-8f87-2bf448d685b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:06.608060 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97355ac-9604-4a8e-8f87-2bf448d685b1 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:06.611692 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b5443-de3a-4fc9-93f7-f6371d50b349 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:06.612307 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b5443-de3a-4fc9-93f7-f6371d50b349 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:06.612795 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b5443-de3a-4fc9-93f7-f6371d50b349 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:06.613178 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b5443-de3a-4fc9-93f7-f6371d50b349 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:06.626991 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211784cf-af70-4503-8164-195583804d98 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:06.627136 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29b5443-de3a-4fc9-93f7-f6371d50b349 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:06.627233 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-211784cf-af70-4503-8164-195583804d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:06.627351 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29b5443-de3a-4fc9-93f7-f6371d50b349 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:06.639357 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211784cf-af70-4503-8164-195583804d98 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:06.651399 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1054/2125] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:51:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:51:06.651533 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ceca8c-c1f9-4230-8741-8e8c7512e9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:06.652182 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ceca8c-c1f9-4230-8741-8e8c7512e9a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:06.652528 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ceca8c-c1f9-4230-8741-8e8c7512e9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:06.652809 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ceca8c-c1f9-4230-8741-8e8c7512e9a8 admin admin] PATH_INFO: `/v3/projects/7f341daf8ff14ce7b31e7ee96a4cef23` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:06.654025 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ceca8c-c1f9-4230-8741-8e8c7512e9a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f341daf8ff14ce7b31e7ee96a4cef23)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:06.656141 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ceca8c-c1f9-4230-8741-8e8c7512e9a8 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:06.658879 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:06.817174 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-62ceca8c-c1f9-4230-8741-8e8c7512e9a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f341daf8ff14ce7b31e7ee96a4cef23', 'request_id': 'req-62ceca8c-c1f9-4230-8741-8e8c7512e9a8'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:06.825565 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-62ceca8c-c1f9-4230-8741-8e8c7512e9a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f341daf8ff14ce7b31e7ee96a4cef23', 'request_id': 'req-62ceca8c-c1f9-4230-8741-8e8c7512e9a8'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:06.905470 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-62ceca8c-c1f9-4230-8741-8e8c7512e9a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f341daf8ff14ce7b31e7ee96a4cef23', 'request_id': 'req-62ceca8c-c1f9-4230-8741-8e8c7512e9a8'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:06.917874 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-62ceca8c-c1f9-4230-8741-8e8c7512e9a8 admin admin] The token cache is being invalidate because project 7f341daf8ff14ce7b31e7ee96a4cef23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:51:06.918168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-62ceca8c-c1f9-4230-8741-8e8c7512e9a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:06.918832 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-62ceca8c-c1f9-4230-8741-8e8c7512e9a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:06.923415 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1072/2126] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:51:06 2022] DELETE /identity/v3/projects/7f341daf8ff14ce7b31e7ee96a4cef23 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:51:06.924410 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b5443-de3a-4fc9-93f7-f6371d50b349 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:06.935127 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b5443-de3a-4fc9-93f7-f6371d50b349 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:06.943345 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:07.001945 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6eeea8-ddfc-451d-b6ac-41191f5c920b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:07.002607 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6eeea8-ddfc-451d-b6ac-41191f5c920b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:07.002889 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6eeea8-ddfc-451d-b6ac-41191f5c920b admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:07.003162 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6eeea8-ddfc-451d-b6ac-41191f5c920b admin admin] PATH_INFO: `/v3/users/c4251514859f4af8bd8fb8baa65499ef` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:07.004481 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6eeea8-ddfc-451d-b6ac-41191f5c920b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4251514859f4af8bd8fb8baa65499ef)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:07.006425 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6eeea8-ddfc-451d-b6ac-41191f5c920b admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:07.554466 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ceca8c-c1f9-4230-8741-8e8c7512e9a8 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:07.556355 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-62ceca8c-c1f9-4230-8741-8e8c7512e9a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:07.557203 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7f6eeea8-ddfc-451d-b6ac-41191f5c920b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4251514859f4af8bd8fb8baa65499ef'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:07.577701 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ceca8c-c1f9-4230-8741-8e8c7512e9a8 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:07.577914 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7f6eeea8-ddfc-451d-b6ac-41191f5c920b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4251514859f4af8bd8fb8baa65499ef'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:07.578687 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7f6eeea8-ddfc-451d-b6ac-41191f5c920b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4251514859f4af8bd8fb8baa65499ef'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:07.587979 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7f6eeea8-ddfc-451d-b6ac-41191f5c920b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4251514859f4af8bd8fb8baa65499ef'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:07.594340 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78edb6cc-9462-4139-a1d2-dda3df676528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:07.595134 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78edb6cc-9462-4139-a1d2-dda3df676528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:07.597112 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78edb6cc-9462-4139-a1d2-dda3df676528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:07.597112 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78edb6cc-9462-4139-a1d2-dda3df676528 admin admin] PATH_INFO: `/v3/projects/e3d8f2d046b34dd197f07c52593b6f60` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:07.600972 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78edb6cc-9462-4139-a1d2-dda3df676528 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3d8f2d046b34dd197f07c52593b6f60)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:07.600972 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78edb6cc-9462-4139-a1d2-dda3df676528 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:07.688486 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7f6eeea8-ddfc-451d-b6ac-41191f5c920b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4251514859f4af8bd8fb8baa65499ef'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:07.696131 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1055/2127] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:51:06 2022] DELETE /identity/v3/users/c4251514859f4af8bd8fb8baa65499ef => generated 0 bytes in 772 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:51:07.720394 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6eeea8-ddfc-451d-b6ac-41191f5c920b admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:07.721885 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-7f6eeea8-ddfc-451d-b6ac-41191f5c920b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:07.759618 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-78edb6cc-9462-4139-a1d2-dda3df676528 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3d8f2d046b34dd197f07c52593b6f60', 'request_id': 'req-78edb6cc-9462-4139-a1d2-dda3df676528'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:07.768959 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-78edb6cc-9462-4139-a1d2-dda3df676528 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3d8f2d046b34dd197f07c52593b6f60', 'request_id': 'req-78edb6cc-9462-4139-a1d2-dda3df676528'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:07.809578 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6eeea8-ddfc-451d-b6ac-41191f5c920b admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:07.849168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-78edb6cc-9462-4139-a1d2-dda3df676528 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3d8f2d046b34dd197f07c52593b6f60', 'request_id': 'req-78edb6cc-9462-4139-a1d2-dda3df676528'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:07.862425 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-78edb6cc-9462-4139-a1d2-dda3df676528 admin admin] The token cache is being invalidate because project e3d8f2d046b34dd197f07c52593b6f60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:51:07.862780 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-78edb6cc-9462-4139-a1d2-dda3df676528 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:07.863540 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-78edb6cc-9462-4139-a1d2-dda3df676528 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:07.865405 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1073/2128] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:51:07 2022] DELETE /identity/v3/projects/e3d8f2d046b34dd197f07c52593b6f60 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:07.870672 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78edb6cc-9462-4139-a1d2-dda3df676528 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:07.872439 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-78edb6cc-9462-4139-a1d2-dda3df676528 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:07.880320 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cd0a30-ea86-47c4-b025-154ff54bac43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:07.880979 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd0a30-ea86-47c4-b025-154ff54bac43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:07.881265 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd0a30-ea86-47c4-b025-154ff54bac43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:07.881539 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd0a30-ea86-47c4-b025-154ff54bac43 admin admin] PATH_INFO: `/v3/users/c7a7804e81be4e76bbbe4ecc223f423d` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:07.882748 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cd0a30-ea86-47c4-b025-154ff54bac43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7a7804e81be4e76bbbe4ecc223f423d)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:07.884727 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cd0a30-ea86-47c4-b025-154ff54bac43 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:07.889197 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78edb6cc-9462-4139-a1d2-dda3df676528 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:07.893653 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c09c7-e91f-4e97-ad20-edbc15564a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:07.895627 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c09c7-e91f-4e97-ad20-edbc15564a06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:07.895983 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c09c7-e91f-4e97-ad20-edbc15564a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:07.896329 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c09c7-e91f-4e97-ad20-edbc15564a06 admin admin] PATH_INFO: `/v3/users/685fa22a04c3440e8934c70698c48717` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:07.897545 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192c09c7-e91f-4e97-ad20-edbc15564a06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=685fa22a04c3440e8934c70698c48717)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:07.899558 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192c09c7-e91f-4e97-ad20-edbc15564a06 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:08.088130 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-33cd0a30-ea86-47c4-b025-154ff54bac43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7a7804e81be4e76bbbe4ecc223f423d'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:08.101757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-33cd0a30-ea86-47c4-b025-154ff54bac43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7a7804e81be4e76bbbe4ecc223f423d'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:08.102556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-33cd0a30-ea86-47c4-b025-154ff54bac43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7a7804e81be4e76bbbe4ecc223f423d'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:08.115367 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-33cd0a30-ea86-47c4-b025-154ff54bac43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7a7804e81be4e76bbbe4ecc223f423d'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:08.143037 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-192c09c7-e91f-4e97-ad20-edbc15564a06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '685fa22a04c3440e8934c70698c48717'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:08.150671 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-192c09c7-e91f-4e97-ad20-edbc15564a06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '685fa22a04c3440e8934c70698c48717'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:08.161115 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=76,keystone:DELETE=22,keystone:INSERT=2 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:08.163400 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-192c09c7-e91f-4e97-ad20-edbc15564a06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '685fa22a04c3440e8934c70698c48717'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:08.164225 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-192c09c7-e91f-4e97-ad20-edbc15564a06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '685fa22a04c3440e8934c70698c48717'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:08.214004 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-33cd0a30-ea86-47c4-b025-154ff54bac43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7a7804e81be4e76bbbe4ecc223f423d'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:08.220470 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1056/2129] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:51:07 2022] DELETE /identity/v3/users/c7a7804e81be4e76bbbe4ecc223f423d => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:51:08.224946 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dcc6c6-2a03-45be-94b4-c9f92bd76d30 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:08.225237 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dcc6c6-2a03-45be-94b4-c9f92bd76d30 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:08.225511 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dcc6c6-2a03-45be-94b4-c9f92bd76d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:08.251261 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-d9dcc6c6-2a03-45be-94b4-c9f92bd76d30 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:08.253057 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d9dcc6c6-2a03-45be-94b4-c9f92bd76d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:08.256458 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-192c09c7-e91f-4e97-ad20-edbc15564a06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '685fa22a04c3440e8934c70698c48717'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:08.271060 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1074/2130] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:51:07 2022] DELETE /identity/v3/users/685fa22a04c3440e8934c70698c48717 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:51:08.271060 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c09c7-e91f-4e97-ad20-edbc15564a06 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:08.271060 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-192c09c7-e91f-4e97-ad20-edbc15564a06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:08.328871 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1057/2131] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:08.339493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dcc6c6-2a03-45be-94b4-c9f92bd76d30 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:08.340592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d9dcc6c6-2a03-45be-94b4-c9f92bd76d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:08.371727 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c09c7-e91f-4e97-ad20-edbc15564a06 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:08.379763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9982d67-dddf-4818-8959-e47295116128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:08.381909 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9982d67-dddf-4818-8959-e47295116128 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:08.382414 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9982d67-dddf-4818-8959-e47295116128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:08.382534 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9982d67-dddf-4818-8959-e47295116128 admin admin] PATH_INFO: `/v3/projects/cf0cbb8172a943ef8ce6e156cbc71eaa` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:08.384330 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9982d67-dddf-4818-8959-e47295116128 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf0cbb8172a943ef8ce6e156cbc71eaa)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:08.391417 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9982d67-dddf-4818-8959-e47295116128 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:08.411138 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dcc6c6-2a03-45be-94b4-c9f92bd76d30 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:08.458364 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b89ba8-ed9d-4dbf-93d7-2821f001b5b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:08.458967 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b89ba8-ed9d-4dbf-93d7-2821f001b5b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:08.459240 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b89ba8-ed9d-4dbf-93d7-2821f001b5b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:08.459539 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b89ba8-ed9d-4dbf-93d7-2821f001b5b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:08.468442 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b89ba8-ed9d-4dbf-93d7-2821f001b5b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:08.469858 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b89ba8-ed9d-4dbf-93d7-2821f001b5b1 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:08.479484 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1058/2132] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:51:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:51:08.580559 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-b9982d67-dddf-4818-8959-e47295116128 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf0cbb8172a943ef8ce6e156cbc71eaa', 'request_id': 'req-b9982d67-dddf-4818-8959-e47295116128'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:08.613739 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-b9982d67-dddf-4818-8959-e47295116128 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf0cbb8172a943ef8ce6e156cbc71eaa', 'request_id': 'req-b9982d67-dddf-4818-8959-e47295116128'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:08.728573 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-b9982d67-dddf-4818-8959-e47295116128 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf0cbb8172a943ef8ce6e156cbc71eaa', 'request_id': 'req-b9982d67-dddf-4818-8959-e47295116128'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:08.741344 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-b9982d67-dddf-4818-8959-e47295116128 admin admin] The token cache is being invalidate because project cf0cbb8172a943ef8ce6e156cbc71eaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:51:08.741759 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-b9982d67-dddf-4818-8959-e47295116128 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:08.742644 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-b9982d67-dddf-4818-8959-e47295116128 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:08.744178 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1075/2133] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:51:08 2022] DELETE /identity/v3/projects/cf0cbb8172a943ef8ce6e156cbc71eaa => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:08.749926 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac2c96-7cc8-4e5f-829f-50061e5b4d82 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:08.750442 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac2c96-7cc8-4e5f-829f-50061e5b4d82 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:08.750442 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac2c96-7cc8-4e5f-829f-50061e5b4d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:08.785505 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-01ac2c96-7cc8-4e5f-829f-50061e5b4d82 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:08.785505 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-01ac2c96-7cc8-4e5f-829f-50061e5b4d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:08.852291 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1059/2134] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:08.861566 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc677a7-0e85-405e-9020-74fa3920e78e None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:08.861744 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc677a7-0e85-405e-9020-74fa3920e78e None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:08.866104 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc677a7-0e85-405e-9020-74fa3920e78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:08.893598 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-afc677a7-0e85-405e-9020-74fa3920e78e None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:08.895549 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-afc677a7-0e85-405e-9020-74fa3920e78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:08.901123 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac2c96-7cc8-4e5f-829f-50061e5b4d82 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:08.902881 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-01ac2c96-7cc8-4e5f-829f-50061e5b4d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:08.909782 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1076/2135] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:08.912927 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc677a7-0e85-405e-9020-74fa3920e78e None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:08.923088 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac2c96-7cc8-4e5f-829f-50061e5b4d82 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:08.931438 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ffc9d-2dae-4134-8fd0-a85e3584bbc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:08.932095 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ffc9d-2dae-4134-8fd0-a85e3584bbc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:08.932375 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ffc9d-2dae-4134-8fd0-a85e3584bbc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:08.932654 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ffc9d-2dae-4134-8fd0-a85e3584bbc6 admin admin] PATH_INFO: `/v3/projects/326a5aacdce14be8aa055816e947c729` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:08.933816 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ffc9d-2dae-4134-8fd0-a85e3584bbc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=326a5aacdce14be8aa055816e947c729)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:08.935760 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ffc9d-2dae-4134-8fd0-a85e3584bbc6 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:08.946635 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc677a7-0e85-405e-9020-74fa3920e78e None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:08.951583 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c113e47-e99f-4d4a-938c-45905f461fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:08.952195 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c113e47-e99f-4d4a-938c-45905f461fcc admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:08.952479 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c113e47-e99f-4d4a-938c-45905f461fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:08.953103 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c113e47-e99f-4d4a-938c-45905f461fcc admin admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:08.953454 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c113e47-e99f-4d4a-938c-45905f461fcc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:51:08.953766 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c113e47-e99f-4d4a-938c-45905f461fcc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:08.955367 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c113e47-e99f-4d4a-938c-45905f461fcc admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:08.968122 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1077/2136] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:51:08.972347 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c113e47-e99f-4d4a-938c-45905f461fcc admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:08.982392 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c113e47-e99f-4d4a-938c-45905f461fcc admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:08.987280 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b988330b-34a5-44da-a74a-248435f26038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:08.988491 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b988330b-34a5-44da-a74a-248435f26038 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:08.988759 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b988330b-34a5-44da-a74a-248435f26038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:08.989031 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b988330b-34a5-44da-a74a-248435f26038 admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:08.990135 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b988330b-34a5-44da-a74a-248435f26038 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:08.991978 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b988330b-34a5-44da-a74a-248435f26038 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:09.136894 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1078/2137] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:51:08 2022] POST /identity/v3/projects => generated 389 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:51:09.136894 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b988330b-34a5-44da-a74a-248435f26038 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:09.142900 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-b988330b-34a5-44da-a74a-248435f26038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:09.153443 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-8b0ffc9d-2dae-4134-8fd0-a85e3584bbc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '326a5aacdce14be8aa055816e947c729', 'request_id': 'req-8b0ffc9d-2dae-4134-8fd0-a85e3584bbc6'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:09.161167 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-8b0ffc9d-2dae-4134-8fd0-a85e3584bbc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '326a5aacdce14be8aa055816e947c729', 'request_id': 'req-8b0ffc9d-2dae-4134-8fd0-a85e3584bbc6'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:09.213631 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b988330b-34a5-44da-a74a-248435f26038 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:09.221053 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-8b0ffc9d-2dae-4134-8fd0-a85e3584bbc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '326a5aacdce14be8aa055816e947c729', 'request_id': 'req-8b0ffc9d-2dae-4134-8fd0-a85e3584bbc6'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:09.236759 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-8b0ffc9d-2dae-4134-8fd0-a85e3584bbc6 admin admin] The token cache is being invalidate because project 326a5aacdce14be8aa055816e947c729 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:51:09.237058 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-8b0ffc9d-2dae-4134-8fd0-a85e3584bbc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:09.237719 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-8b0ffc9d-2dae-4134-8fd0-a85e3584bbc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:09.243069 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1060/2138] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:51:08 2022] DELETE /identity/v3/projects/326a5aacdce14be8aa055816e947c729 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:09.243069 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ffc9d-2dae-4134-8fd0-a85e3584bbc6 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:09.243069 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-8b0ffc9d-2dae-4134-8fd0-a85e3584bbc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:09.297003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dc1e3b-b26e-41bf-8fb5-6c425f9a2a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:09.297746 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc1e3b-b26e-41bf-8fb5-6c425f9a2a83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:09.298147 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc1e3b-b26e-41bf-8fb5-6c425f9a2a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:09.298512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc1e3b-b26e-41bf-8fb5-6c425f9a2a83 admin admin] PATH_INFO: `/v3/projects/ad6748e5b9094b4c8ef92209e458abaf` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:09.299763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dc1e3b-b26e-41bf-8fb5-6c425f9a2a83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad6748e5b9094b4c8ef92209e458abaf)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:09.305137 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dc1e3b-b26e-41bf-8fb5-6c425f9a2a83 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:09.323037 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ffc9d-2dae-4134-8fd0-a85e3584bbc6 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:09.336523 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade6fd2b-ab20-4e0d-b9b3-b03bbf877e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:09.337065 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6fd2b-ab20-4e0d-b9b3-b03bbf877e03 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:09.337291 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6fd2b-ab20-4e0d-b9b3-b03bbf877e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:09.337600 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6fd2b-ab20-4e0d-b9b3-b03bbf877e03 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:09.338256 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade6fd2b-ab20-4e0d-b9b3-b03bbf877e03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:09.340257 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade6fd2b-ab20-4e0d-b9b3-b03bbf877e03 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:09.428541 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1061/2139] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:51:09 2022] POST /identity/v3/users => generated 295 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:51:09.429568 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade6fd2b-ab20-4e0d-b9b3-b03bbf877e03 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:09.432184 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ade6fd2b-ab20-4e0d-b9b3-b03bbf877e03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:09.444897 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade6fd2b-ab20-4e0d-b9b3-b03bbf877e03 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:09.447907 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-46dc1e3b-b26e-41bf-8fb5-6c425f9a2a83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad6748e5b9094b4c8ef92209e458abaf', 'request_id': 'req-46dc1e3b-b26e-41bf-8fb5-6c425f9a2a83'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:09.455076 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-46dc1e3b-b26e-41bf-8fb5-6c425f9a2a83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad6748e5b9094b4c8ef92209e458abaf', 'request_id': 'req-46dc1e3b-b26e-41bf-8fb5-6c425f9a2a83'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:09.506453 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:09.507504 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca1408-faac-46bf-9ad3-6488e3009060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:09.508104 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca1408-faac-46bf-9ad3-6488e3009060 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:09.508382 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca1408-faac-46bf-9ad3-6488e3009060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:09.508649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca1408-faac-46bf-9ad3-6488e3009060 admin admin] PATH_INFO: `/v3/users/c71f14c86d5d47fbbdac63e857b782f2` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:09.509884 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ca1408-faac-46bf-9ad3-6488e3009060 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c71f14c86d5d47fbbdac63e857b782f2)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:09.511840 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ca1408-faac-46bf-9ad3-6488e3009060 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:09.545172 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-46dc1e3b-b26e-41bf-8fb5-6c425f9a2a83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad6748e5b9094b4c8ef92209e458abaf', 'request_id': 'req-46dc1e3b-b26e-41bf-8fb5-6c425f9a2a83'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:09.557041 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-46dc1e3b-b26e-41bf-8fb5-6c425f9a2a83 admin admin] The token cache is being invalidate because project ad6748e5b9094b4c8ef92209e458abaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:51:09.557339 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-46dc1e3b-b26e-41bf-8fb5-6c425f9a2a83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:09.558001 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-46dc1e3b-b26e-41bf-8fb5-6c425f9a2a83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:09.559557 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1079/2140] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:51:09 2022] DELETE /identity/v3/projects/ad6748e5b9094b4c8ef92209e458abaf => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:09.562537 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dc1e3b-b26e-41bf-8fb5-6c425f9a2a83 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:09.562537 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-46dc1e3b-b26e-41bf-8fb5-6c425f9a2a83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:09.574124 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dc1e3b-b26e-41bf-8fb5-6c425f9a2a83 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:09.578985 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50493e1-fd9f-46f1-af6b-1b1573a5867f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:09.580527 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50493e1-fd9f-46f1-af6b-1b1573a5867f admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:09.580527 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50493e1-fd9f-46f1-af6b-1b1573a5867f admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:09.580527 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50493e1-fd9f-46f1-af6b-1b1573a5867f admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:09.581296 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50493e1-fd9f-46f1-af6b-1b1573a5867f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:09.582959 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50493e1-fd9f-46f1-af6b-1b1573a5867f admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:09.593220 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1080/2141] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:51:09.593835 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50493e1-fd9f-46f1-af6b-1b1573a5867f admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:09.595498 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-c50493e1-fd9f-46f1-af6b-1b1573a5867f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:09.608681 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50493e1-fd9f-46f1-af6b-1b1573a5867f admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:09.613481 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a86a741-b696-44a0-89e7-f4ee0c81492a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:09.615019 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a86a741-b696-44a0-89e7-f4ee0c81492a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:09.615136 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a86a741-b696-44a0-89e7-f4ee0c81492a admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:09.615441 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a86a741-b696-44a0-89e7-f4ee0c81492a admin admin] PATH_INFO: `/v3/users/e80af2d74ff9428e841b95352fc16f9e` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:09.616589 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a86a741-b696-44a0-89e7-f4ee0c81492a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e80af2d74ff9428e841b95352fc16f9e)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:09.618521 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a86a741-b696-44a0-89e7-f4ee0c81492a admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:09.702397 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a6ca1408-faac-46bf-9ad3-6488e3009060 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c71f14c86d5d47fbbdac63e857b782f2'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:09.715665 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a6ca1408-faac-46bf-9ad3-6488e3009060 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c71f14c86d5d47fbbdac63e857b782f2'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:09.716514 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a6ca1408-faac-46bf-9ad3-6488e3009060 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c71f14c86d5d47fbbdac63e857b782f2'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:09.724539 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a6ca1408-faac-46bf-9ad3-6488e3009060 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c71f14c86d5d47fbbdac63e857b782f2'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:09.786806 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a6ca1408-faac-46bf-9ad3-6488e3009060 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c71f14c86d5d47fbbdac63e857b782f2'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:09.797579 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1062/2142] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:51:09 2022] DELETE /identity/v3/users/c71f14c86d5d47fbbdac63e857b782f2 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:09.808025 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca1408-faac-46bf-9ad3-6488e3009060 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:09.809262 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-6a86a741-b696-44a0-89e7-f4ee0c81492a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e80af2d74ff9428e841b95352fc16f9e'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:09.814324 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a6ca1408-faac-46bf-9ad3-6488e3009060 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:09.819642 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-6a86a741-b696-44a0-89e7-f4ee0c81492a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e80af2d74ff9428e841b95352fc16f9e'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:09.835807 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-6a86a741-b696-44a0-89e7-f4ee0c81492a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e80af2d74ff9428e841b95352fc16f9e'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:09.836921 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-6a86a741-b696-44a0-89e7-f4ee0c81492a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e80af2d74ff9428e841b95352fc16f9e'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:09.907278 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca1408-faac-46bf-9ad3-6488e3009060 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:09.914162 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc07526-3de7-42fb-aa8d-450e7c3138f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:09.914812 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc07526-3de7-42fb-aa8d-450e7c3138f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:09.915091 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc07526-3de7-42fb-aa8d-450e7c3138f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:09.915387 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc07526-3de7-42fb-aa8d-450e7c3138f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:09.916042 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc07526-3de7-42fb-aa8d-450e7c3138f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:09.917627 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc07526-3de7-42fb-aa8d-450e7c3138f8 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:09.928285 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1063/2143] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:51:09.932772 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc07526-3de7-42fb-aa8d-450e7c3138f8 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:09.934430 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-3bc07526-3de7-42fb-aa8d-450e7c3138f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:09.946131 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-6a86a741-b696-44a0-89e7-f4ee0c81492a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e80af2d74ff9428e841b95352fc16f9e'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:09.947484 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc07526-3de7-42fb-aa8d-450e7c3138f8 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:09.955068 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033aa99d-8254-49a4-accf-58a0b84db944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:09.955761 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033aa99d-8254-49a4-accf-58a0b84db944 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:09.956040 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033aa99d-8254-49a4-accf-58a0b84db944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:09.956317 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033aa99d-8254-49a4-accf-58a0b84db944 admin admin] PATH_INFO: `/v3/projects/55d56aa662684aef99e02942c56b3e9e/users/9f831ee08c654a5b9fd63598f3fa8c9e/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:09.957619 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1081/2144] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:51:09 2022] DELETE /identity/v3/users/e80af2d74ff9428e841b95352fc16f9e => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:09.983696 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033aa99d-8254-49a4-accf-58a0b84db944 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55d56aa662684aef99e02942c56b3e9e, user_id=9f831ee08c654a5b9fd63598f3fa8c9e, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:09.986428 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033aa99d-8254-49a4-accf-58a0b84db944 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:10.194953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1064/2145] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:51:09 2022] PUT /identity/v3/projects/55d56aa662684aef99e02942c56b3e9e/users/9f831ee08c654a5b9fd63598f3fa8c9e/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:51:10.209768 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:10.214019 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec08cb-e36d-416a-9e1d-3f4072e1d9dc None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:10.214320 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec08cb-e36d-416a-9e1d-3f4072e1d9dc None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:10.214589 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec08cb-e36d-416a-9e1d-3f4072e1d9dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:10.246822 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-a7ec08cb-e36d-416a-9e1d-3f4072e1d9dc None None] MFA Rules not processed for user `9f831ee08c654a5b9fd63598f3fa8c9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:10.248143 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-a7ec08cb-e36d-416a-9e1d-3f4072e1d9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:10.357908 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1082/2146] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:10 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:10.377219 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033aa99d-8254-49a4-accf-58a0b84db944 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:10.385874 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-033aa99d-8254-49a4-accf-58a0b84db944 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:10.486529 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033aa99d-8254-49a4-accf-58a0b84db944 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:10.490962 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6703f1a0-343a-4034-bfaf-d427319b04df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:10.492415 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703f1a0-343a-4034-bfaf-d427319b04df service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:10.492696 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703f1a0-343a-4034-bfaf-d427319b04df service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:10.495800 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703f1a0-343a-4034-bfaf-d427319b04df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:10.505715 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6703f1a0-343a-4034-bfaf-d427319b04df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:10.507279 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6703f1a0-343a-4034-bfaf-d427319b04df service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:10.516720 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1065/2147] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:51:10 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:51:10.600505 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ec08cb-e36d-416a-9e1d-3f4072e1d9dc None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:10.601006 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6703f1a0-343a-4034-bfaf-d427319b04df service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:10.602665 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-6703f1a0-343a-4034-bfaf-d427319b04df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:10.602993 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-a7ec08cb-e36d-416a-9e1d-3f4072e1d9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:10.704994 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6703f1a0-343a-4034-bfaf-d427319b04df service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:10.711550 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f59de-17f8-4c95-b3c7-b529888ee963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:10.716109 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f59de-17f8-4c95-b3c7-b529888ee963 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:10.716109 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f59de-17f8-4c95-b3c7-b529888ee963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:10.716109 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f59de-17f8-4c95-b3c7-b529888ee963 admin admin] PATH_INFO: `/v3/projects/99c124db1b694dc790e9a0062fa72d26` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:10.716109 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696f59de-17f8-4c95-b3c7-b529888ee963 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99c124db1b694dc790e9a0062fa72d26)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:10.716109 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696f59de-17f8-4c95-b3c7-b529888ee963 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:10.731207 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ec08cb-e36d-416a-9e1d-3f4072e1d9dc None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:10.741186 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758f19c-148e-4a8b-8bf0-ea9c3ef37f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:10.742308 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758f19c-148e-4a8b-8bf0-ea9c3ef37f8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:10.742668 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758f19c-148e-4a8b-8bf0-ea9c3ef37f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:10.742947 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758f19c-148e-4a8b-8bf0-ea9c3ef37f8f admin admin] PATH_INFO: `/v3/projects/62a7ed828e1b467ba65da373e1e97331` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:10.744369 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758f19c-148e-4a8b-8bf0-ea9c3ef37f8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=62a7ed828e1b467ba65da373e1e97331)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:10.746316 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758f19c-148e-4a8b-8bf0-ea9c3ef37f8f admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:10.854258 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-696f59de-17f8-4c95-b3c7-b529888ee963 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99c124db1b694dc790e9a0062fa72d26', 'request_id': 'req-696f59de-17f8-4c95-b3c7-b529888ee963'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:10.855123 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-6758f19c-148e-4a8b-8bf0-ea9c3ef37f8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62a7ed828e1b467ba65da373e1e97331', 'request_id': 'req-6758f19c-148e-4a8b-8bf0-ea9c3ef37f8f'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:10.860345 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-696f59de-17f8-4c95-b3c7-b529888ee963 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99c124db1b694dc790e9a0062fa72d26', 'request_id': 'req-696f59de-17f8-4c95-b3c7-b529888ee963'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:10.867460 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-6758f19c-148e-4a8b-8bf0-ea9c3ef37f8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62a7ed828e1b467ba65da373e1e97331', 'request_id': 'req-6758f19c-148e-4a8b-8bf0-ea9c3ef37f8f'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:10.922803 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-696f59de-17f8-4c95-b3c7-b529888ee963 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99c124db1b694dc790e9a0062fa72d26', 'request_id': 'req-696f59de-17f8-4c95-b3c7-b529888ee963'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:10.936685 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-696f59de-17f8-4c95-b3c7-b529888ee963 admin admin] The token cache is being invalidate because project 99c124db1b694dc790e9a0062fa72d26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:51:10.936977 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-696f59de-17f8-4c95-b3c7-b529888ee963 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:10.937637 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-696f59de-17f8-4c95-b3c7-b529888ee963 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:10.939181 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1066/2148] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:51:10 2022] DELETE /identity/v3/projects/99c124db1b694dc790e9a0062fa72d26 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:10.944855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe3364-66fb-4bdd-947d-048b4fa9dae6 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:10.945078 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe3364-66fb-4bdd-947d-048b4fa9dae6 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:10.945354 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe3364-66fb-4bdd-947d-048b4fa9dae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:10.966173 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-3efe3364-66fb-4bdd-947d-048b4fa9dae6 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:10.967444 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-3efe3364-66fb-4bdd-947d-048b4fa9dae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:10.968253 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-6758f19c-148e-4a8b-8bf0-ea9c3ef37f8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62a7ed828e1b467ba65da373e1e97331', 'request_id': 'req-6758f19c-148e-4a8b-8bf0-ea9c3ef37f8f'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:10.980099 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-6758f19c-148e-4a8b-8bf0-ea9c3ef37f8f admin admin] The token cache is being invalidate because project 62a7ed828e1b467ba65da373e1e97331 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:51:10.981784 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-6758f19c-148e-4a8b-8bf0-ea9c3ef37f8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:10.981784 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-6758f19c-148e-4a8b-8bf0-ea9c3ef37f8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:10.982459 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1083/2149] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:51:10 2022] DELETE /identity/v3/projects/62a7ed828e1b467ba65da373e1e97331 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:10.989484 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8cbb50-77e1-44e2-b0dc-3e06642a8767 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:10.989771 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8cbb50-77e1-44e2-b0dc-3e06642a8767 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:10.990041 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8cbb50-77e1-44e2-b0dc-3e06642a8767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:11.009871 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-5e8cbb50-77e1-44e2-b0dc-3e06642a8767 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:11.012343 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-5e8cbb50-77e1-44e2-b0dc-3e06642a8767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:11.027880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1067/2150] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:51:11.035487 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2899d-8bf1-41dc-84ab-fc7d0ec47bbc None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:11.035715 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2899d-8bf1-41dc-84ab-fc7d0ec47bbc None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:11.035991 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2899d-8bf1-41dc-84ab-fc7d0ec47bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:11.055638 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-9cc2899d-8bf1-41dc-84ab-fc7d0ec47bbc None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:11.058215 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-9cc2899d-8bf1-41dc-84ab-fc7d0ec47bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:11.065076 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1068/2151] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:51:11.069862 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc2899d-8bf1-41dc-84ab-fc7d0ec47bbc None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:11.072639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1084/2152] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:51:11.078949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc2899d-8bf1-41dc-84ab-fc7d0ec47bbc None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:11.081230 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5e6e1-b67e-4ff9-85ad-70d402a818a1 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:11.081230 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5e6e1-b67e-4ff9-85ad-70d402a818a1 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:11.081572 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5e6e1-b67e-4ff9-85ad-70d402a818a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:11.085370 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b68b40-20a4-47ab-b376-bddf76f972f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:11.086179 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b68b40-20a4-47ab-b376-bddf76f972f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:11.086286 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b68b40-20a4-47ab-b376-bddf76f972f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:11.086492 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b68b40-20a4-47ab-b376-bddf76f972f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:11.087164 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b68b40-20a4-47ab-b376-bddf76f972f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:51:11.087493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b68b40-20a4-47ab-b376-bddf76f972f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:11.089092 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b68b40-20a4-47ab-b376-bddf76f972f8 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:11.101802 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-64f5e6e1-b67e-4ff9-85ad-70d402a818a1 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:11.105672 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1069/2153] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:51:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:51:11.105797 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-64f5e6e1-b67e-4ff9-85ad-70d402a818a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:11.106492 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b68b40-20a4-47ab-b376-bddf76f972f8 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:11.115364 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1085/2154] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:11.117409 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b68b40-20a4-47ab-b376-bddf76f972f8 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:11.119142 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f5e6e1-b67e-4ff9-85ad-70d402a818a1 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:11.122517 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f089d-de9d-4e30-8734-cd1cab3ec931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:11.123211 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f089d-de9d-4e30-8734-cd1cab3ec931 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:11.123448 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f089d-de9d-4e30-8734-cd1cab3ec931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:11.123774 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f089d-de9d-4e30-8734-cd1cab3ec931 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:11.124507 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997f089d-de9d-4e30-8734-cd1cab3ec931 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:11.126532 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997f089d-de9d-4e30-8734-cd1cab3ec931 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:11.130404 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f5e6e1-b67e-4ff9-85ad-70d402a818a1 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:11.135398 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44d494-8c70-4a36-a4e4-74b0358a2b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:11.139186 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44d494-8c70-4a36-a4e4-74b0358a2b16 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:11.139186 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44d494-8c70-4a36-a4e4-74b0358a2b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:11.139186 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44d494-8c70-4a36-a4e4-74b0358a2b16 admin admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:11.139186 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e44d494-8c70-4a36-a4e4-74b0358a2b16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:51:11.139186 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e44d494-8c70-4a36-a4e4-74b0358a2b16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:11.139186 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e44d494-8c70-4a36-a4e4-74b0358a2b16 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:11.151834 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1086/2155] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:51:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:51:11.156272 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44d494-8c70-4a36-a4e4-74b0358a2b16 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:11.167730 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44d494-8c70-4a36-a4e4-74b0358a2b16 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:11.179577 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4817820-e541-473c-9d86-6d473a2c3b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:11.179577 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4817820-e541-473c-9d86-6d473a2c3b79 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:11.179577 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4817820-e541-473c-9d86-6d473a2c3b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:11.179577 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4817820-e541-473c-9d86-6d473a2c3b79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:11.179988 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4817820-e541-473c-9d86-6d473a2c3b79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:11.179988 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4817820-e541-473c-9d86-6d473a2c3b79 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:11.186058 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:11.265149 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1070/2156] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:51:11 2022] POST /identity/v3/projects => generated 383 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:51:11.266191 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f089d-de9d-4e30-8734-cd1cab3ec931 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:11.295783 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1087/2157] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:51:11 2022] POST /identity/v3/projects => generated 409 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:51:11.296671 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4817820-e541-473c-9d86-6d473a2c3b79 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:11.353094 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f089d-de9d-4e30-8734-cd1cab3ec931 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:11.382617 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4817820-e541-473c-9d86-6d473a2c3b79 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:11.387798 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbf1833-0074-4ef1-a533-d06e261573f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:11.388484 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf1833-0074-4ef1-a533-d06e261573f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:11.388736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf1833-0074-4ef1-a533-d06e261573f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:11.389010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf1833-0074-4ef1-a533-d06e261573f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:11.389737 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbf1833-0074-4ef1-a533-d06e261573f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:11.392609 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbf1833-0074-4ef1-a533-d06e261573f1 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:11.430219 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e762f012-ab72-4826-9dc3-1c152cc0690e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:11.430976 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762f012-ab72-4826-9dc3-1c152cc0690e admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:11.431260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762f012-ab72-4826-9dc3-1c152cc0690e admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:11.431642 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762f012-ab72-4826-9dc3-1c152cc0690e admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:11.432292 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e762f012-ab72-4826-9dc3-1c152cc0690e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:11.434239 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e762f012-ab72-4826-9dc3-1c152cc0690e admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:11.540690 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1071/2158] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:51:11 2022] POST /identity/v3/users => generated 292 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:51:11.544604 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1088/2159] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:51:11 2022] POST /identity/v3/users => generated 305 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:51:11.545104 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e762f012-ab72-4826-9dc3-1c152cc0690e admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:11.548365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbf1833-0074-4ef1-a533-d06e261573f1 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:11.559416 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbf1833-0074-4ef1-a533-d06e261573f1 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:11.562523 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e762f012-ab72-4826-9dc3-1c152cc0690e admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:11.564301 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2324df6-4a3a-4f09-8da1-13d516b7bdec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:11.564904 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2324df6-4a3a-4f09-8da1-13d516b7bdec admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:11.565184 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2324df6-4a3a-4f09-8da1-13d516b7bdec admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:11.565457 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2324df6-4a3a-4f09-8da1-13d516b7bdec admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:11.566764 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2324df6-4a3a-4f09-8da1-13d516b7bdec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:11.568432 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2324df6-4a3a-4f09-8da1-13d516b7bdec admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:11.569105 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a32fa8b-54ac-4f2a-9608-bc085b766cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:11.569998 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32fa8b-54ac-4f2a-9608-bc085b766cf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:11.570431 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32fa8b-54ac-4f2a-9608-bc085b766cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:11.571001 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32fa8b-54ac-4f2a-9608-bc085b766cf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:11.571851 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a32fa8b-54ac-4f2a-9608-bc085b766cf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:11.573556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a32fa8b-54ac-4f2a-9608-bc085b766cf5 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:11.580078 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1089/2160] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 12:51:11.585383 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2324df6-4a3a-4f09-8da1-13d516b7bdec admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:11.590003 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1072/2161] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 12:51:11.591553 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a32fa8b-54ac-4f2a-9608-bc085b766cf5 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:11.595853 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2324df6-4a3a-4f09-8da1-13d516b7bdec admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:11.601791 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91748fac-0030-419e-99e5-9dc62d995e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:11.602370 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a32fa8b-54ac-4f2a-9608-bc085b766cf5 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:11.602480 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91748fac-0030-419e-99e5-9dc62d995e91 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:11.602680 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91748fac-0030-419e-99e5-9dc62d995e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:11.603047 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91748fac-0030-419e-99e5-9dc62d995e91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:11.606348 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91748fac-0030-419e-99e5-9dc62d995e91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:11.606348 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91748fac-0030-419e-99e5-9dc62d995e91 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:11.607560 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f5cc2d-c2c7-47e4-8aec-9e131b96ca74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:11.608253 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5cc2d-c2c7-47e4-8aec-9e131b96ca74 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:11.609680 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5cc2d-c2c7-47e4-8aec-9e131b96ca74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:11.609680 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5cc2d-c2c7-47e4-8aec-9e131b96ca74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:11.609680 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f5cc2d-c2c7-47e4-8aec-9e131b96ca74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:11.611450 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f5cc2d-c2c7-47e4-8aec-9e131b96ca74 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:11.619922 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1090/2162] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:51:11.624555 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91748fac-0030-419e-99e5-9dc62d995e91 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:11.627304 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1073/2163] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 12:51:11.637922 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91748fac-0030-419e-99e5-9dc62d995e91 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:11.642554 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f5cc2d-c2c7-47e4-8aec-9e131b96ca74 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:11.643728 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a68d1-7f0e-4db2-a7aa-dafdfe791e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:11.644352 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a68d1-7f0e-4db2-a7aa-dafdfe791e7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:11.644610 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a68d1-7f0e-4db2-a7aa-dafdfe791e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:11.644953 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a68d1-7f0e-4db2-a7aa-dafdfe791e7f admin admin] PATH_INFO: `/v3/projects/060d08042858403292f4b6629f65a1de/users/c5ed263a54dc47988b665d1fa6579262/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:11.653950 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f5cc2d-c2c7-47e4-8aec-9e131b96ca74 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:11.658940 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c964b1e-a71d-4086-9dea-bf6b678f5579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:11.660209 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c964b1e-a71d-4086-9dea-bf6b678f5579 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:11.663827 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c964b1e-a71d-4086-9dea-bf6b678f5579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:11.663827 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c964b1e-a71d-4086-9dea-bf6b678f5579 admin admin] PATH_INFO: `/v3/projects/9a3352f10a214bf1b948592116cb6f1b/users/85e7d2bc63cb4626984b3e4643560d71/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:11.664061 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4a68d1-7f0e-4db2-a7aa-dafdfe791e7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=060d08042858403292f4b6629f65a1de, user_id=c5ed263a54dc47988b665d1fa6579262, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:11.665278 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4a68d1-7f0e-4db2-a7aa-dafdfe791e7f admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:11.676503 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c964b1e-a71d-4086-9dea-bf6b678f5579 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a3352f10a214bf1b948592116cb6f1b, user_id=85e7d2bc63cb4626984b3e4643560d71, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:11.679155 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c964b1e-a71d-4086-9dea-bf6b678f5579 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:11.797404 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1091/2164] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:51:11 2022] PUT /identity/v3/projects/060d08042858403292f4b6629f65a1de/users/c5ed263a54dc47988b665d1fa6579262/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:51:11.804461 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1074/2165] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:51:11 2022] PUT /identity/v3/projects/9a3352f10a214bf1b948592116cb6f1b/users/85e7d2bc63cb4626984b3e4643560d71/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:11.810216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be477855-95f8-44cd-88d0-3489e2e6f43b None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:11.810382 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e539bd-1626-419e-a37e-3e58e7bff09a None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:11.810509 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be477855-95f8-44cd-88d0-3489e2e6f43b None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:11.810605 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e539bd-1626-419e-a37e-3e58e7bff09a None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:11.810785 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be477855-95f8-44cd-88d0-3489e2e6f43b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:11.810885 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e539bd-1626-419e-a37e-3e58e7bff09a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:11.838053 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-be477855-95f8-44cd-88d0-3489e2e6f43b None None] MFA Rules not processed for user `85e7d2bc63cb4626984b3e4643560d71`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:11.839260 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-be477855-95f8-44cd-88d0-3489e2e6f43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:11.839804 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-a4e539bd-1626-419e-a37e-3e58e7bff09a None None] MFA Rules not processed for user `c5ed263a54dc47988b665d1fa6579262`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:11.841023 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a4e539bd-1626-419e-a37e-3e58e7bff09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:11.915860 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1092/2166] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:11.922419 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1075/2167] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:11 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:11.922907 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be477855-95f8-44cd-88d0-3489e2e6f43b None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:11.923376 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e539bd-1626-419e-a37e-3e58e7bff09a None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:12.010755 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be477855-95f8-44cd-88d0-3489e2e6f43b None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:12.021399 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c7dfc8-10d9-429c-9e5e-d18e4d2f352d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:12.022225 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c7dfc8-10d9-429c-9e5e-d18e4d2f352d admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:12.022583 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c7dfc8-10d9-429c-9e5e-d18e4d2f352d admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:12.023114 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c7dfc8-10d9-429c-9e5e-d18e4d2f352d admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:12.023971 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c7dfc8-10d9-429c-9e5e-d18e4d2f352d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:12.032049 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e539bd-1626-419e-a37e-3e58e7bff09a None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:12.033193 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c7dfc8-10d9-429c-9e5e-d18e4d2f352d admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:12.038711 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43aa22a-fafb-4d92-801e-4596e0379edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:12.041040 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43aa22a-fafb-4d92-801e-4596e0379edc admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:12.041040 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43aa22a-fafb-4d92-801e-4596e0379edc admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:12.041040 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43aa22a-fafb-4d92-801e-4596e0379edc admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:12.041040 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43aa22a-fafb-4d92-801e-4596e0379edc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:12.042617 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43aa22a-fafb-4d92-801e-4596e0379edc admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:12.166130 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1076/2168] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:51:11 2022] POST /identity/v3/projects => generated 409 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:51:12.167781 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1093/2169] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:51:11 2022] POST /identity/v3/projects => generated 383 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:51:12.171725 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c7dfc8-10d9-429c-9e5e-d18e4d2f352d admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:12.171894 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43aa22a-fafb-4d92-801e-4596e0379edc admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:12.245041 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c7dfc8-10d9-429c-9e5e-d18e4d2f352d admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:12.245566 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43aa22a-fafb-4d92-801e-4596e0379edc admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:12.255380 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5977deb2-907d-47ab-85de-9a7b5f16fef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:12.256203 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5977deb2-907d-47ab-85de-9a7b5f16fef3 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:12.256489 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5977deb2-907d-47ab-85de-9a7b5f16fef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:12.256602 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc976f-ef07-4a17-b5c5-07ce31f14b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:12.256796 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5977deb2-907d-47ab-85de-9a7b5f16fef3 admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:12.257288 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc976f-ef07-4a17-b5c5-07ce31f14b4a admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:12.257462 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5977deb2-907d-47ab-85de-9a7b5f16fef3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:12.257566 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc976f-ef07-4a17-b5c5-07ce31f14b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:12.258453 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc976f-ef07-4a17-b5c5-07ce31f14b4a admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:12.259306 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbc976f-ef07-4a17-b5c5-07ce31f14b4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:12.259524 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5977deb2-907d-47ab-85de-9a7b5f16fef3 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:12.261372 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbc976f-ef07-4a17-b5c5-07ce31f14b4a admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:12.339880 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:12.341588 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1077/2170] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:51:12 2022] POST /identity/v3/users => generated 311 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:51:12.345601 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc976f-ef07-4a17-b5c5-07ce31f14b4a admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:12.349211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1094/2171] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:51:12 2022] POST /identity/v3/users => generated 298 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:51:12.350309 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5977deb2-907d-47ab-85de-9a7b5f16fef3 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:12.363620 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5977deb2-907d-47ab-85de-9a7b5f16fef3 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:12.369417 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc976f-ef07-4a17-b5c5-07ce31f14b4a admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:12.369417 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdc4ea3-c0e1-4423-973a-c046b6475457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:12.369979 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc4ea3-c0e1-4423-973a-c046b6475457 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:12.370256 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc4ea3-c0e1-4423-973a-c046b6475457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:12.370530 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc4ea3-c0e1-4423-973a-c046b6475457 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:12.371169 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdc4ea3-c0e1-4423-973a-c046b6475457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:12.372480 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c3349c-84d2-4c35-9770-91819dc760ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:12.372852 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdc4ea3-c0e1-4423-973a-c046b6475457 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:12.373493 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c3349c-84d2-4c35-9770-91819dc760ce admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:12.373964 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c3349c-84d2-4c35-9770-91819dc760ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:12.374429 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c3349c-84d2-4c35-9770-91819dc760ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:12.375423 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c3349c-84d2-4c35-9770-91819dc760ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:12.377160 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c3349c-84d2-4c35-9770-91819dc760ce admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:12.386921 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1078/2172] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:12 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 12:51:12.391058 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdc4ea3-c0e1-4423-973a-c046b6475457 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:12.392218 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1095/2173] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:12 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:51:12.394329 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c3349c-84d2-4c35-9770-91819dc760ce admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:12.404230 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c3349c-84d2-4c35-9770-91819dc760ce admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:12.424716 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdc4ea3-c0e1-4423-973a-c046b6475457 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:12.424870 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7584b01-bb8d-490c-9e64-6829b1084c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:12.424870 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7584b01-bb8d-490c-9e64-6829b1084c3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:12.424870 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7584b01-bb8d-490c-9e64-6829b1084c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:12.424870 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7584b01-bb8d-490c-9e64-6829b1084c3a admin admin] PATH_INFO: `/v3/projects/d3d86efbcbad422796c41a4c2f2e4b98/users/1b9a2b8126234b4881b4d56408c26ba5/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:12.425195 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7cf55-e7e2-4da3-9f9f-8b2f3956a0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:12.425195 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7cf55-e7e2-4da3-9f9f-8b2f3956a0f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:12.425195 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7cf55-e7e2-4da3-9f9f-8b2f3956a0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:12.425195 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7cf55-e7e2-4da3-9f9f-8b2f3956a0f5 admin admin] PATH_INFO: `/v3/projects/63d62693a972457cac2c2834a93a9b23/users/7979f68c683b447aa4cf82470adb0eeb/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:12.427257 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7584b01-bb8d-490c-9e64-6829b1084c3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3d86efbcbad422796c41a4c2f2e4b98, user_id=1b9a2b8126234b4881b4d56408c26ba5, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:12.430305 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7584b01-bb8d-490c-9e64-6829b1084c3a admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:12.431704 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d7cf55-e7e2-4da3-9f9f-8b2f3956a0f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63d62693a972457cac2c2834a93a9b23, user_id=7979f68c683b447aa4cf82470adb0eeb, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:12.434423 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d7cf55-e7e2-4da3-9f9f-8b2f3956a0f5 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:12.546741 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1079/2174] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:51:12 2022] PUT /identity/v3/projects/63d62693a972457cac2c2834a93a9b23/users/7979f68c683b447aa4cf82470adb0eeb/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:51:12.549728 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1096/2175] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:51:12 2022] PUT /identity/v3/projects/d3d86efbcbad422796c41a4c2f2e4b98/users/1b9a2b8126234b4881b4d56408c26ba5/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:51:12.551697 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7cf55-e7e2-4da3-9f9f-8b2f3956a0f5 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:12.554328 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7584b01-bb8d-490c-9e64-6829b1084c3a admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:12.637332 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7584b01-bb8d-490c-9e64-6829b1084c3a admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:12.638293 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7cf55-e7e2-4da3-9f9f-8b2f3956a0f5 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:12.642829 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9554a8f-c810-44f1-9cc6-4092d7ad1be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:12.643480 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac96e6-6a7d-420a-b4ab-79ee3621b852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:12.643612 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9554a8f-c810-44f1-9cc6-4092d7ad1be4 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:12.643804 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9554a8f-c810-44f1-9cc6-4092d7ad1be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:12.644119 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9554a8f-c810-44f1-9cc6-4092d7ad1be4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:12.648526 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac96e6-6a7d-420a-b4ab-79ee3621b852 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:12.648526 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac96e6-6a7d-420a-b4ab-79ee3621b852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:12.648526 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac96e6-6a7d-420a-b4ab-79ee3621b852 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:12.648875 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9554a8f-c810-44f1-9cc6-4092d7ad1be4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:12.648969 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ac96e6-6a7d-420a-b4ab-79ee3621b852 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:12.648969 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ac96e6-6a7d-420a-b4ab-79ee3621b852 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:12.649146 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9554a8f-c810-44f1-9cc6-4092d7ad1be4 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:12.658956 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1080/2176] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:12 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:51:12.659418 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1097/2177] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:12 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:51:12.664036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac96e6-6a7d-420a-b4ab-79ee3621b852 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:12.664300 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9554a8f-c810-44f1-9cc6-4092d7ad1be4 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:12.674036 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9554a8f-c810-44f1-9cc6-4092d7ad1be4 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:12.674393 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac96e6-6a7d-420a-b4ab-79ee3621b852 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:12.679365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63fba3f-7070-49a2-b8ce-af5022322c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:12.679758 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d582b0fe-27c8-492a-88c1-b890855d0bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:12.680020 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63fba3f-7070-49a2-b8ce-af5022322c39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:12.680324 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63fba3f-7070-49a2-b8ce-af5022322c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:12.680559 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582b0fe-27c8-492a-88c1-b890855d0bea admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:12.680664 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63fba3f-7070-49a2-b8ce-af5022322c39 admin admin] PATH_INFO: `/v3/projects/63d62693a972457cac2c2834a93a9b23/users/7979f68c683b447aa4cf82470adb0eeb/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:12.680837 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582b0fe-27c8-492a-88c1-b890855d0bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:12.681110 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582b0fe-27c8-492a-88c1-b890855d0bea admin admin] PATH_INFO: `/v3/projects/d3d86efbcbad422796c41a4c2f2e4b98/users/1b9a2b8126234b4881b4d56408c26ba5/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:12.682729 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63fba3f-7070-49a2-b8ce-af5022322c39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63d62693a972457cac2c2834a93a9b23, user_id=7979f68c683b447aa4cf82470adb0eeb, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:12.683014 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d582b0fe-27c8-492a-88c1-b890855d0bea admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3d86efbcbad422796c41a4c2f2e4b98, user_id=1b9a2b8126234b4881b4d56408c26ba5, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:12.685390 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63fba3f-7070-49a2-b8ce-af5022322c39 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:12.685688 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d582b0fe-27c8-492a-88c1-b890855d0bea admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:12.696166 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1098/2178] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:51:12 2022] PUT /identity/v3/projects/63d62693a972457cac2c2834a93a9b23/users/7979f68c683b447aa4cf82470adb0eeb/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:51:12.699604 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1081/2179] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:51:12 2022] PUT /identity/v3/projects/d3d86efbcbad422796c41a4c2f2e4b98/users/1b9a2b8126234b4881b4d56408c26ba5/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:12.707029 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af23e478-d9f8-4781-bf4a-eacf5f6d9e49 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:12.707545 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af23e478-d9f8-4781-bf4a-eacf5f6d9e49 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:12.707826 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af23e478-d9f8-4781-bf4a-eacf5f6d9e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:12.709846 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a1aed-9a1c-45ec-b642-79ec4afc108b None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:12.709968 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a1aed-9a1c-45ec-b642-79ec4afc108b None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:12.710214 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a1aed-9a1c-45ec-b642-79ec4afc108b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:12.739852 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-af23e478-d9f8-4781-bf4a-eacf5f6d9e49 None None] MFA Rules not processed for user `7979f68c683b447aa4cf82470adb0eeb`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:12.741073 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-af23e478-d9f8-4781-bf4a-eacf5f6d9e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:12.742210 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-0e1a1aed-9a1c-45ec-b642-79ec4afc108b None None] MFA Rules not processed for user `1b9a2b8126234b4881b4d56408c26ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:12.743513 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-0e1a1aed-9a1c-45ec-b642-79ec4afc108b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:12.812874 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1082/2180] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:12 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:51:12.813253 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1099/2181] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:12 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:12.824408 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a1aed-9a1c-45ec-b642-79ec4afc108b None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:12.825764 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-0e1a1aed-9a1c-45ec-b642-79ec4afc108b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:12.863453 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af23e478-d9f8-4781-bf4a-eacf5f6d9e49 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:12.865767 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-af23e478-d9f8-4781-bf4a-eacf5f6d9e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:12.890500 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a1aed-9a1c-45ec-b642-79ec4afc108b None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:12.894707 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5e981-1cbc-4182-a2ad-e0fa2c613da0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:12.895339 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5e981-1cbc-4182-a2ad-e0fa2c613da0 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:12.895638 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5e981-1cbc-4182-a2ad-e0fa2c613da0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:12.895910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5e981-1cbc-4182-a2ad-e0fa2c613da0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:12.905102 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef5e981-1cbc-4182-a2ad-e0fa2c613da0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:12.906527 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef5e981-1cbc-4182-a2ad-e0fa2c613da0 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:12.916986 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1083/2182] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:51:12 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:51:12.929495 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af23e478-d9f8-4781-bf4a-eacf5f6d9e49 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:12.933796 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ee4e0-5ec1-4038-aa0e-d3c136fc2343 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:12.934637 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ee4e0-5ec1-4038-aa0e-d3c136fc2343 service cinder] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:12.936269 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ee4e0-5ec1-4038-aa0e-d3c136fc2343 service cinder] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:12.936555 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ee4e0-5ec1-4038-aa0e-d3c136fc2343 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:12.947065 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6ee4e0-5ec1-4038-aa0e-d3c136fc2343 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:12.948523 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6ee4e0-5ec1-4038-aa0e-d3c136fc2343 service cinder] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:12.958141 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1100/2183] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:51:12 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:51:13.116271 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5e981-1cbc-4182-a2ad-e0fa2c613da0 service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:13.136994 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5e981-1cbc-4182-a2ad-e0fa2c613da0 service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:13.136994 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b32c8d4-1a1a-41b7-8b12-a508953c9a70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:13.137268 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32c8d4-1a1a-41b7-8b12-a508953c9a70 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:13.137554 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32c8d4-1a1a-41b7-8b12-a508953c9a70 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:13.137855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32c8d4-1a1a-41b7-8b12-a508953c9a70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:13.186323 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:13.200832 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b32c8d4-1a1a-41b7-8b12-a508953c9a70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:13.202418 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b32c8d4-1a1a-41b7-8b12-a508953c9a70 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:13.212160 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1084/2184] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:51:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:51:13.336106 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ee4e0-5ec1-4038-aa0e-d3c136fc2343 service cinder] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:13.344582 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ee4e0-5ec1-4038-aa0e-d3c136fc2343 service cinder] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:13.348768 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365794f5-0d96-43bf-80b3-d0244fa05d58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:13.349380 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365794f5-0d96-43bf-80b3-d0244fa05d58 service cinder] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:13.349661 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365794f5-0d96-43bf-80b3-d0244fa05d58 service cinder] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:13.349934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365794f5-0d96-43bf-80b3-d0244fa05d58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:13.420677 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365794f5-0d96-43bf-80b3-d0244fa05d58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:13.429183 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365794f5-0d96-43bf-80b3-d0244fa05d58 service cinder] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:13.432315 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1101/2185] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:51:13 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:51:13.834556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b32c8d4-1a1a-41b7-8b12-a508953c9a70 service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:13.898101 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b32c8d4-1a1a-41b7-8b12-a508953c9a70 service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:13.902600 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17973548-94ce-42ed-9f1d-00008d97c145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:13.903223 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17973548-94ce-42ed-9f1d-00008d97c145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:13.903531 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17973548-94ce-42ed-9f1d-00008d97c145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:13.903888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17973548-94ce-42ed-9f1d-00008d97c145 admin admin] PATH_INFO: `/v3/users/85e7d2bc63cb4626984b3e4643560d71` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:13.904907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17973548-94ce-42ed-9f1d-00008d97c145 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85e7d2bc63cb4626984b3e4643560d71)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:13.906852 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17973548-94ce-42ed-9f1d-00008d97c145 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:13.939935 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:14.086539 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-17973548-94ce-42ed-9f1d-00008d97c145 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85e7d2bc63cb4626984b3e4643560d71'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:14.099725 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-17973548-94ce-42ed-9f1d-00008d97c145 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85e7d2bc63cb4626984b3e4643560d71'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:14.100414 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-17973548-94ce-42ed-9f1d-00008d97c145 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85e7d2bc63cb4626984b3e4643560d71'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:14.107300 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-17973548-94ce-42ed-9f1d-00008d97c145 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85e7d2bc63cb4626984b3e4643560d71'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:14.260305 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-17973548-94ce-42ed-9f1d-00008d97c145 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85e7d2bc63cb4626984b3e4643560d71'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:14.267366 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1085/2186] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:51:13 2022] DELETE /identity/v3/users/85e7d2bc63cb4626984b3e4643560d71 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:14.273196 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb6171-ef73-45c8-abc4-559fd258a7cf None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:14.278411 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb6171-ef73-45c8-abc4-559fd258a7cf None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:14.278411 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb6171-ef73-45c8-abc4-559fd258a7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:14.336656 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-a0cb6171-ef73-45c8-abc4-559fd258a7cf None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:14.338961 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-a0cb6171-ef73-45c8-abc4-559fd258a7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:14.402061 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1102/2187] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:14.412331 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17973548-94ce-42ed-9f1d-00008d97c145 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:14.414006 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-17973548-94ce-42ed-9f1d-00008d97c145 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:14.478818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17973548-94ce-42ed-9f1d-00008d97c145 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:14.482795 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dea8515-21d5-41ed-80c4-afc1d508a579 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:14.483447 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea8515-21d5-41ed-80c4-afc1d508a579 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:14.483722 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea8515-21d5-41ed-80c4-afc1d508a579 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:14.483989 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea8515-21d5-41ed-80c4-afc1d508a579 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:14.493928 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dea8515-21d5-41ed-80c4-afc1d508a579 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:14.495358 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dea8515-21d5-41ed-80c4-afc1d508a579 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:14.505794 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1086/2188] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:51:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:51:14.666207 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:14.969843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cb6171-ef73-45c8-abc4-559fd258a7cf None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:14.971411 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-a0cb6171-ef73-45c8-abc4-559fd258a7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:14.987418 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cb6171-ef73-45c8-abc4-559fd258a7cf None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:14.994554 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39955fcc-4e24-4a16-a59f-a809fe8c580c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:14.995199 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39955fcc-4e24-4a16-a59f-a809fe8c580c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:14.995518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39955fcc-4e24-4a16-a59f-a809fe8c580c admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:14.995935 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39955fcc-4e24-4a16-a59f-a809fe8c580c admin admin] PATH_INFO: `/v3/projects/9a3352f10a214bf1b948592116cb6f1b` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:14.996988 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39955fcc-4e24-4a16-a59f-a809fe8c580c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a3352f10a214bf1b948592116cb6f1b)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:14.998964 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39955fcc-4e24-4a16-a59f-a809fe8c580c admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:15.008475 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:DELETE=1,keystone:SELECT=94,keystone:INSERT=5 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:15.094511 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-39955fcc-4e24-4a16-a59f-a809fe8c580c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a3352f10a214bf1b948592116cb6f1b', 'request_id': 'req-39955fcc-4e24-4a16-a59f-a809fe8c580c'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:15.101470 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-39955fcc-4e24-4a16-a59f-a809fe8c580c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a3352f10a214bf1b948592116cb6f1b', 'request_id': 'req-39955fcc-4e24-4a16-a59f-a809fe8c580c'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:15.162138 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-39955fcc-4e24-4a16-a59f-a809fe8c580c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a3352f10a214bf1b948592116cb6f1b', 'request_id': 'req-39955fcc-4e24-4a16-a59f-a809fe8c580c'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:15.168731 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:15.172668 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-39955fcc-4e24-4a16-a59f-a809fe8c580c admin admin] The token cache is being invalidate because project 9a3352f10a214bf1b948592116cb6f1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:51:15.172956 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-39955fcc-4e24-4a16-a59f-a809fe8c580c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:15.173613 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-39955fcc-4e24-4a16-a59f-a809fe8c580c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:15.175275 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1103/2189] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:51:14 2022] DELETE /identity/v3/projects/9a3352f10a214bf1b948592116cb6f1b => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:15.179473 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dea8515-21d5-41ed-80c4-afc1d508a579 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:15.189066 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dea8515-21d5-41ed-80c4-afc1d508a579 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:15.249064 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e294dd58-719e-4918-be51-3d4cf55a2864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:15.249845 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294dd58-719e-4918-be51-3d4cf55a2864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:15.250125 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294dd58-719e-4918-be51-3d4cf55a2864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:15.250398 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294dd58-719e-4918-be51-3d4cf55a2864 admin admin] PATH_INFO: `/v3/users/1b9a2b8126234b4881b4d56408c26ba5` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:15.251613 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e294dd58-719e-4918-be51-3d4cf55a2864 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b9a2b8126234b4881b4d56408c26ba5)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:15.256474 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e294dd58-719e-4918-be51-3d4cf55a2864 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:15.265682 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:15.404979 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:15.413900 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e294dd58-719e-4918-be51-3d4cf55a2864 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b9a2b8126234b4881b4d56408c26ba5'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:15.426820 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e294dd58-719e-4918-be51-3d4cf55a2864 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b9a2b8126234b4881b4d56408c26ba5'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:15.427613 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e294dd58-719e-4918-be51-3d4cf55a2864 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b9a2b8126234b4881b4d56408c26ba5'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:15.435010 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e294dd58-719e-4918-be51-3d4cf55a2864 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b9a2b8126234b4881b4d56408c26ba5'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:15.479692 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:15.568950 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e294dd58-719e-4918-be51-3d4cf55a2864 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b9a2b8126234b4881b4d56408c26ba5'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:15.575747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1087/2190] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:51:15 2022] DELETE /identity/v3/users/1b9a2b8126234b4881b4d56408c26ba5 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:15.905584 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:15.985685 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:16.061530 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39955fcc-4e24-4a16-a59f-a809fe8c580c admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:16.062986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-39955fcc-4e24-4a16-a59f-a809fe8c580c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:16.084352 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39955fcc-4e24-4a16-a59f-a809fe8c580c admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:16.148357 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35069395-2499-4a00-835b-8b5b3447c2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:16.149041 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35069395-2499-4a00-835b-8b5b3447c2e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:16.149315 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35069395-2499-4a00-835b-8b5b3447c2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:16.149586 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35069395-2499-4a00-835b-8b5b3447c2e9 admin admin] PATH_INFO: `/v3/projects/d3d86efbcbad422796c41a4c2f2e4b98` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:16.150682 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35069395-2499-4a00-835b-8b5b3447c2e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3d86efbcbad422796c41a4c2f2e4b98)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:16.152626 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35069395-2499-4a00-835b-8b5b3447c2e9 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:16.242964 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:16.250531 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-35069395-2499-4a00-835b-8b5b3447c2e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3d86efbcbad422796c41a4c2f2e4b98', 'request_id': 'req-35069395-2499-4a00-835b-8b5b3447c2e9'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:16.257099 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-35069395-2499-4a00-835b-8b5b3447c2e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3d86efbcbad422796c41a4c2f2e4b98', 'request_id': 'req-35069395-2499-4a00-835b-8b5b3447c2e9'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:16.356783 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-35069395-2499-4a00-835b-8b5b3447c2e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3d86efbcbad422796c41a4c2f2e4b98', 'request_id': 'req-35069395-2499-4a00-835b-8b5b3447c2e9'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:16.374458 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-35069395-2499-4a00-835b-8b5b3447c2e9 admin admin] The token cache is being invalidate because project d3d86efbcbad422796c41a4c2f2e4b98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:51:16.374458 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-35069395-2499-4a00-835b-8b5b3447c2e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:16.374458 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-35069395-2499-4a00-835b-8b5b3447c2e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:16.374458 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1104/2191] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:51:16 2022] DELETE /identity/v3/projects/d3d86efbcbad422796c41a4c2f2e4b98 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:51:16.376887 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d821a-3ff4-4b24-a1ce-31bc85ac9453 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:16.377173 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d821a-3ff4-4b24-a1ce-31bc85ac9453 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:16.377443 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d821a-3ff4-4b24-a1ce-31bc85ac9453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:16.397702 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-df5d821a-3ff4-4b24-a1ce-31bc85ac9453 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:16.403096 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-df5d821a-3ff4-4b24-a1ce-31bc85ac9453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:16.460763 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1088/2192] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:16.468619 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e96ab-b7c1-40f1-afbe-858d22303b0f None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:16.468909 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e96ab-b7c1-40f1-afbe-858d22303b0f None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:16.469187 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e96ab-b7c1-40f1-afbe-858d22303b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:16.491065 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-2a3e96ab-b7c1-40f1-afbe-858d22303b0f None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:16.492316 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-2a3e96ab-b7c1-40f1-afbe-858d22303b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:16.501663 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1105/2193] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:16.506178 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d821a-3ff4-4b24-a1ce-31bc85ac9453 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:16.516062 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d821a-3ff4-4b24-a1ce-31bc85ac9453 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:16.521692 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975044d5-a108-47a5-99aa-753f65da25d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:16.522363 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975044d5-a108-47a5-99aa-753f65da25d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:16.522643 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975044d5-a108-47a5-99aa-753f65da25d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:16.522917 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975044d5-a108-47a5-99aa-753f65da25d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:16.523615 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975044d5-a108-47a5-99aa-753f65da25d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:51:16.523915 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975044d5-a108-47a5-99aa-753f65da25d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:16.525543 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975044d5-a108-47a5-99aa-753f65da25d0 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:16.538225 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1089/2194] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:51:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:51:16.542077 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e96ab-b7c1-40f1-afbe-858d22303b0f None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:16.551616 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e96ab-b7c1-40f1-afbe-858d22303b0f None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:16.556540 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffcf760-8213-403d-ac56-468282195750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:16.557154 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffcf760-8213-403d-ac56-468282195750 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:16.557436 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffcf760-8213-403d-ac56-468282195750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:16.557714 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffcf760-8213-403d-ac56-468282195750 admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:16.558346 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffcf760-8213-403d-ac56-468282195750 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:16.560291 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffcf760-8213-403d-ac56-468282195750 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:16.637767 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:16.691104 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1106/2195] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:51:16 2022] POST /identity/v3/projects => generated 383 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:51:16.695220 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975044d5-a108-47a5-99aa-753f65da25d0 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:16.707271 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975044d5-a108-47a5-99aa-753f65da25d0 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:16.766696 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7e2d13-e974-4e62-8dd3-50baa8ec95ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:16.767333 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e2d13-e974-4e62-8dd3-50baa8ec95ec admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:16.767645 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e2d13-e974-4e62-8dd3-50baa8ec95ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:16.767917 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e2d13-e974-4e62-8dd3-50baa8ec95ec admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:16.768553 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7e2d13-e974-4e62-8dd3-50baa8ec95ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:16.771142 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7e2d13-e974-4e62-8dd3-50baa8ec95ec admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:16.879887 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1090/2196] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:51:16 2022] POST /identity/v3/users => generated 292 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:51:16.882767 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffcf760-8213-403d-ac56-468282195750 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:16.895539 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffcf760-8213-403d-ac56-468282195750 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:16.900455 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f8c0e6-7581-4099-b5c1-34be92978ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:16.901065 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8c0e6-7581-4099-b5c1-34be92978ec6 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:16.901343 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8c0e6-7581-4099-b5c1-34be92978ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:16.901611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8c0e6-7581-4099-b5c1-34be92978ec6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:16.902249 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f8c0e6-7581-4099-b5c1-34be92978ec6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:16.904070 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f8c0e6-7581-4099-b5c1-34be92978ec6 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:16.914481 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1107/2197] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:51:16.918588 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7e2d13-e974-4e62-8dd3-50baa8ec95ec admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:16.928742 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7e2d13-e974-4e62-8dd3-50baa8ec95ec admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:16.933699 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e091b-806a-4097-b64d-160b0d399092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:16.934301 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e091b-806a-4097-b64d-160b0d399092 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:16.934582 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e091b-806a-4097-b64d-160b0d399092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:16.934856 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e091b-806a-4097-b64d-160b0d399092 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:16.935540 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5e091b-806a-4097-b64d-160b0d399092 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:16.937124 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5e091b-806a-4097-b64d-160b0d399092 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:16.951987 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1091/2198] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:51:16.956163 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f8c0e6-7581-4099-b5c1-34be92978ec6 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:16.967522 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f8c0e6-7581-4099-b5c1-34be92978ec6 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:16.972128 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc5d6f-3b76-4ec1-8295-b1a1ec5eebd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:16.973039 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc5d6f-3b76-4ec1-8295-b1a1ec5eebd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:16.973323 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc5d6f-3b76-4ec1-8295-b1a1ec5eebd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:16.973586 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc5d6f-3b76-4ec1-8295-b1a1ec5eebd4 admin admin] PATH_INFO: `/v3/projects/b22cd583306248a1b2f052414af818f1/users/090335bc919c4d3b873c6d482e464033/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:16.987762 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cc5d6f-3b76-4ec1-8295-b1a1ec5eebd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b22cd583306248a1b2f052414af818f1, user_id=090335bc919c4d3b873c6d482e464033, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:16.990806 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cc5d6f-3b76-4ec1-8295-b1a1ec5eebd4 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:17.008405 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:17.065451 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1108/2199] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:51:16 2022] PUT /identity/v3/projects/b22cd583306248a1b2f052414af818f1/users/090335bc919c4d3b873c6d482e464033/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:17.077432 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141ea48-430e-407e-8a01-36da7166eae5 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:17.077716 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141ea48-430e-407e-8a01-36da7166eae5 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:17.078070 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141ea48-430e-407e-8a01-36da7166eae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:17.107520 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-b141ea48-430e-407e-8a01-36da7166eae5 None None] MFA Rules not processed for user `090335bc919c4d3b873c6d482e464033`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:17.109988 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b141ea48-430e-407e-8a01-36da7166eae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:17.174392 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1092/2200] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:17.179759 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc5d6f-3b76-4ec1-8295-b1a1ec5eebd4 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:17.208812 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:17.214496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:17.252984 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc5d6f-3b76-4ec1-8295-b1a1ec5eebd4 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:17.258537 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4103bab-f47e-4dc1-a295-5a184f965a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:17.259145 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4103bab-f47e-4dc1-a295-5a184f965a59 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:17.259468 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4103bab-f47e-4dc1-a295-5a184f965a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:17.259745 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4103bab-f47e-4dc1-a295-5a184f965a59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:17.260383 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4103bab-f47e-4dc1-a295-5a184f965a59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:17.262346 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4103bab-f47e-4dc1-a295-5a184f965a59 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:17.359188 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1109/2201] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:51:17 2022] POST /identity/v3/projects => generated 385 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:51:17.364102 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141ea48-430e-407e-8a01-36da7166eae5 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:17.374983 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141ea48-430e-407e-8a01-36da7166eae5 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:17.432724 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a506ad-4d59-495b-950d-3d79c3278d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:17.433649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a506ad-4d59-495b-950d-3d79c3278d12 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:17.433930 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a506ad-4d59-495b-950d-3d79c3278d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:17.434196 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a506ad-4d59-495b-950d-3d79c3278d12 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:17.434840 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a506ad-4d59-495b-950d-3d79c3278d12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:17.436935 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a506ad-4d59-495b-950d-3d79c3278d12 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:17.541029 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1093/2202] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:51:17 2022] POST /identity/v3/users => generated 299 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:51:17.545045 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4103bab-f47e-4dc1-a295-5a184f965a59 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:17.555744 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4103bab-f47e-4dc1-a295-5a184f965a59 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:17.560325 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d03ce-fb51-4617-99b1-c41388e30dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:17.560920 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d03ce-fb51-4617-99b1-c41388e30dff admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:17.561201 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d03ce-fb51-4617-99b1-c41388e30dff admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:17.561477 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d03ce-fb51-4617-99b1-c41388e30dff admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:17.562110 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5d03ce-fb51-4617-99b1-c41388e30dff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:17.563741 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5d03ce-fb51-4617-99b1-c41388e30dff admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:17.575374 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1110/2203] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:51:17.582906 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a506ad-4d59-495b-950d-3d79c3278d12 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:17.592859 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a506ad-4d59-495b-950d-3d79c3278d12 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:17.598216 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d8e93-987a-499e-97c6-3f2e8cb4eb00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:17.598791 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d8e93-987a-499e-97c6-3f2e8cb4eb00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:17.599068 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d8e93-987a-499e-97c6-3f2e8cb4eb00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:17.599364 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d8e93-987a-499e-97c6-3f2e8cb4eb00 admin admin] PATH_INFO: `/v3/projects/fd710dd7ca454d819aaf50dc7fc7372f/users/96fe6b68eaf14e8bab52f257ad7b556c/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:17.615150 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67d8e93-987a-499e-97c6-3f2e8cb4eb00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd710dd7ca454d819aaf50dc7fc7372f, user_id=96fe6b68eaf14e8bab52f257ad7b556c, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:17.617805 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67d8e93-987a-499e-97c6-3f2e8cb4eb00 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:17.688180 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1094/2204] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:51:17 2022] PUT /identity/v3/projects/fd710dd7ca454d819aaf50dc7fc7372f/users/96fe6b68eaf14e8bab52f257ad7b556c/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:51:17.692174 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d03ce-fb51-4617-99b1-c41388e30dff admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:17.702388 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d03ce-fb51-4617-99b1-c41388e30dff admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:17.768142 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddf30c8-abe6-4b23-8a25-a25ac49b92c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:17.768760 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf30c8-abe6-4b23-8a25-a25ac49b92c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:17.769045 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf30c8-abe6-4b23-8a25-a25ac49b92c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:17.770036 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf30c8-abe6-4b23-8a25-a25ac49b92c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:17.770708 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddf30c8-abe6-4b23-8a25-a25ac49b92c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:17.772324 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddf30c8-abe6-4b23-8a25-a25ac49b92c8 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:17.782471 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1111/2205] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:17 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 12:51:17.787623 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d8e93-987a-499e-97c6-3f2e8cb4eb00 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:17.798134 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d8e93-987a-499e-97c6-3f2e8cb4eb00 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:17.804817 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9816de47-f832-4c2f-8102-00312694aa21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:17.804817 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816de47-f832-4c2f-8102-00312694aa21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:17.805101 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816de47-f832-4c2f-8102-00312694aa21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:17.805430 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816de47-f832-4c2f-8102-00312694aa21 admin admin] PATH_INFO: `/v3/projects/fd710dd7ca454d819aaf50dc7fc7372f/users/96fe6b68eaf14e8bab52f257ad7b556c/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:17.807631 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9816de47-f832-4c2f-8102-00312694aa21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd710dd7ca454d819aaf50dc7fc7372f, user_id=96fe6b68eaf14e8bab52f257ad7b556c, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:17.810246 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9816de47-f832-4c2f-8102-00312694aa21 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:17.821140 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1095/2206] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:51:17 2022] PUT /identity/v3/projects/fd710dd7ca454d819aaf50dc7fc7372f/users/96fe6b68eaf14e8bab52f257ad7b556c/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:17.831438 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d10cbb-0822-4885-b2e9-33fdb06c442b None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:17.831805 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d10cbb-0822-4885-b2e9-33fdb06c442b None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:17.832428 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d10cbb-0822-4885-b2e9-33fdb06c442b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:17.860421 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-24d10cbb-0822-4885-b2e9-33fdb06c442b None None] MFA Rules not processed for user `96fe6b68eaf14e8bab52f257ad7b556c`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:17.861625 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-24d10cbb-0822-4885-b2e9-33fdb06c442b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:17.939240 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1112/2207] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:17 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:17.950246 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9816de47-f832-4c2f-8102-00312694aa21 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:17.951963 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-9816de47-f832-4c2f-8102-00312694aa21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:18.014798 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9816de47-f832-4c2f-8102-00312694aa21 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:18.018456 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a62186e-90a1-45c6-93ae-7dcccb2ace3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:18.019068 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62186e-90a1-45c6-93ae-7dcccb2ace3e service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:18.019896 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62186e-90a1-45c6-93ae-7dcccb2ace3e service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:18.020166 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62186e-90a1-45c6-93ae-7dcccb2ace3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:18.046719 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:18.082312 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a62186e-90a1-45c6-93ae-7dcccb2ace3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:18.083836 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a62186e-90a1-45c6-93ae-7dcccb2ace3e service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:18.094520 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1096/2208] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:51:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:51:23.530989 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b86c3-3ea3-4a3d-ba6b-4a890cd4545f None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:23.531533 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b86c3-3ea3-4a3d-ba6b-4a890cd4545f None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:23.531645 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b86c3-3ea3-4a3d-ba6b-4a890cd4545f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:23.552839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-a59b86c3-3ea3-4a3d-ba6b-4a890cd4545f None None] MFA Rules not processed for user `78b322344df24e8ca513c19361b79257`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:23.554118 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-a59b86c3-3ea3-4a3d-ba6b-4a890cd4545f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:23.567918 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:23.745156 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1113/2209] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:28.145873 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=17 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:32.167446 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a62186e-90a1-45c6-93ae-7dcccb2ace3e service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:32.168913 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4a62186e-90a1-45c6-93ae-7dcccb2ace3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:32.251113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a62186e-90a1-45c6-93ae-7dcccb2ace3e service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:32.258042 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f50402b-0eab-4bb4-b650-1bd266c3a9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:32.263547 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50402b-0eab-4bb4-b650-1bd266c3a9e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:32.263547 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50402b-0eab-4bb4-b650-1bd266c3a9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:32.263547 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50402b-0eab-4bb4-b650-1bd266c3a9e7 admin admin] PATH_INFO: `/v3/users/c5ed263a54dc47988b665d1fa6579262` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:32.263547 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f50402b-0eab-4bb4-b650-1bd266c3a9e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5ed263a54dc47988b665d1fa6579262)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:32.263547 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f50402b-0eab-4bb4-b650-1bd266c3a9e7 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:32.329805 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59b86c3-3ea3-4a3d-ba6b-4a890cd4545f None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:32.334003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-a59b86c3-3ea3-4a3d-ba6b-4a890cd4545f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:32.349100 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59b86c3-3ea3-4a3d-ba6b-4a890cd4545f None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:32.353925 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486db45d-0f3e-4ed4-89b6-28d556bdd2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:32.354710 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486db45d-0f3e-4ed4-89b6-28d556bdd2f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:32.354992 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486db45d-0f3e-4ed4-89b6-28d556bdd2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:32.355293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486db45d-0f3e-4ed4-89b6-28d556bdd2f3 admin admin] PATH_INFO: `/v3/users/1375c2dded8b4ea1a3ae9730b2525b5b` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:32.356519 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486db45d-0f3e-4ed4-89b6-28d556bdd2f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1375c2dded8b4ea1a3ae9730b2525b5b)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:32.358593 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486db45d-0f3e-4ed4-89b6-28d556bdd2f3 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:32.490950 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4f50402b-0eab-4bb4-b650-1bd266c3a9e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5ed263a54dc47988b665d1fa6579262'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:32.505238 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4f50402b-0eab-4bb4-b650-1bd266c3a9e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5ed263a54dc47988b665d1fa6579262'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:32.506006 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4f50402b-0eab-4bb4-b650-1bd266c3a9e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5ed263a54dc47988b665d1fa6579262'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:32.515254 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4f50402b-0eab-4bb4-b650-1bd266c3a9e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5ed263a54dc47988b665d1fa6579262'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:32.637777 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4f50402b-0eab-4bb4-b650-1bd266c3a9e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5ed263a54dc47988b665d1fa6579262'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:32.640266 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-486db45d-0f3e-4ed4-89b6-28d556bdd2f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1375c2dded8b4ea1a3ae9730b2525b5b'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:32.650688 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-486db45d-0f3e-4ed4-89b6-28d556bdd2f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1375c2dded8b4ea1a3ae9730b2525b5b'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:32.654563 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1097/2210] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:51:32 2022] DELETE /identity/v3/users/c5ed263a54dc47988b665d1fa6579262 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:32.656226 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f50402b-0eab-4bb4-b650-1bd266c3a9e7 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:32.658361 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4f50402b-0eab-4bb4-b650-1bd266c3a9e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:32.668465 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-486db45d-0f3e-4ed4-89b6-28d556bdd2f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1375c2dded8b4ea1a3ae9730b2525b5b'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:32.669671 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-486db45d-0f3e-4ed4-89b6-28d556bdd2f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1375c2dded8b4ea1a3ae9730b2525b5b'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:32.756563 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f50402b-0eab-4bb4-b650-1bd266c3a9e7 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:32.760189 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a90412-c49c-4d3e-8869-2b8e2c864803 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:32.760835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a90412-c49c-4d3e-8869-2b8e2c864803 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:32.761113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a90412-c49c-4d3e-8869-2b8e2c864803 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:32.761386 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a90412-c49c-4d3e-8869-2b8e2c864803 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:32.763218 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-77a90412-c49c-4d3e-8869-2b8e2c864803 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:32.763358 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-486db45d-0f3e-4ed4-89b6-28d556bdd2f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1375c2dded8b4ea1a3ae9730b2525b5b'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:32.770620 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1114/2211] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:51:32 2022] DELETE /identity/v3/users/1375c2dded8b4ea1a3ae9730b2525b5b => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:51:32.774780 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82236b-1f7d-4d39-a17c-f67ba93eef9c None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:32.775068 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82236b-1f7d-4d39-a17c-f67ba93eef9c None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:32.775387 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82236b-1f7d-4d39-a17c-f67ba93eef9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:32.794032 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-5c82236b-1f7d-4d39-a17c-f67ba93eef9c None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:32.795290 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-5c82236b-1f7d-4d39-a17c-f67ba93eef9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:32.798208 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:32.834856 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a90412-c49c-4d3e-8869-2b8e2c864803 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:32.837526 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a90412-c49c-4d3e-8869-2b8e2c864803 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:32.862032 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1098/2212] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:51:32 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:51:32.864434 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4f131-8f4f-4109-922e-c72a230a4d8c None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:32.864718 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4f131-8f4f-4109-922e-c72a230a4d8c None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:32.865413 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4f131-8f4f-4109-922e-c72a230a4d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:32.883095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1115/2213] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:32.893508 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-63f4f131-8f4f-4109-922e-c72a230a4d8c None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:32.893911 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c82236b-1f7d-4d39-a17c-f67ba93eef9c None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:32.896612 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-63f4f131-8f4f-4109-922e-c72a230a4d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:32.900125 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-5c82236b-1f7d-4d39-a17c-f67ba93eef9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:32.906650 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1099/2214] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:32.917152 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f4f131-8f4f-4109-922e-c72a230a4d8c None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:32.918685 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-63f4f131-8f4f-4109-922e-c72a230a4d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:32.995676 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:33.025024 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f4f131-8f4f-4109-922e-c72a230a4d8c None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:33.025600 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c82236b-1f7d-4d39-a17c-f67ba93eef9c None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:33.028773 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb27cf-7d18-4080-8afe-de3380d212ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:33.032485 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb27cf-7d18-4080-8afe-de3380d212ca service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:33.032485 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb27cf-7d18-4080-8afe-de3380d212ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:33.032485 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb27cf-7d18-4080-8afe-de3380d212ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:33.032765 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b375f4-1adb-4f34-a42e-ba77ef80d4d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:33.032765 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b375f4-1adb-4f34-a42e-ba77ef80d4d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:33.032765 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b375f4-1adb-4f34-a42e-ba77ef80d4d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:33.032765 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b375f4-1adb-4f34-a42e-ba77ef80d4d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:33.040829 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb27cf-7d18-4080-8afe-de3380d212ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:33.042361 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb27cf-7d18-4080-8afe-de3380d212ca service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:33.056478 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1100/2215] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:51:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:51:33.065603 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b375f4-1adb-4f34-a42e-ba77ef80d4d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:33.068765 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b375f4-1adb-4f34-a42e-ba77ef80d4d4 service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:33.081790 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1116/2216] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:51:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:51:33.651725 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb27cf-7d18-4080-8afe-de3380d212ca service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:33.657290 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-edfb27cf-7d18-4080-8afe-de3380d212ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:33.696821 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb27cf-7d18-4080-8afe-de3380d212ca service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:33.705107 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859fec7a-c10b-48b4-950c-eb6e99d1b972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:33.706358 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fec7a-c10b-48b4-950c-eb6e99d1b972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:33.708027 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fec7a-c10b-48b4-950c-eb6e99d1b972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:33.708468 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fec7a-c10b-48b4-950c-eb6e99d1b972 admin admin] PATH_INFO: `/v3/projects/060d08042858403292f4b6629f65a1de` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:33.712061 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859fec7a-c10b-48b4-950c-eb6e99d1b972 admin admin] RBAC: Authorizing `identity:delete_project(project_id=060d08042858403292f4b6629f65a1de)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:33.717132 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859fec7a-c10b-48b4-950c-eb6e99d1b972 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:33.848343 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-859fec7a-c10b-48b4-950c-eb6e99d1b972 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '060d08042858403292f4b6629f65a1de', 'request_id': 'req-859fec7a-c10b-48b4-950c-eb6e99d1b972'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:33.854708 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-859fec7a-c10b-48b4-950c-eb6e99d1b972 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '060d08042858403292f4b6629f65a1de', 'request_id': 'req-859fec7a-c10b-48b4-950c-eb6e99d1b972'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:33.921088 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b375f4-1adb-4f34-a42e-ba77ef80d4d4 service neutron] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:33.922518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-08b375f4-1adb-4f34-a42e-ba77ef80d4d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:33.934328 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-859fec7a-c10b-48b4-950c-eb6e99d1b972 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '060d08042858403292f4b6629f65a1de', 'request_id': 'req-859fec7a-c10b-48b4-950c-eb6e99d1b972'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:33.954946 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b375f4-1adb-4f34-a42e-ba77ef80d4d4 service neutron] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:33.955121 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-859fec7a-c10b-48b4-950c-eb6e99d1b972 admin admin] The token cache is being invalidate because project 060d08042858403292f4b6629f65a1de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:51:33.955350 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-859fec7a-c10b-48b4-950c-eb6e99d1b972 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:33.956057 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-859fec7a-c10b-48b4-950c-eb6e99d1b972 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:33.958194 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1101/2217] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:51:33 2022] DELETE /identity/v3/projects/060d08042858403292f4b6629f65a1de => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:33.962319 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859fec7a-c10b-48b4-950c-eb6e99d1b972 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:33.963976 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-859fec7a-c10b-48b4-950c-eb6e99d1b972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:34.019520 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f61e7ab-b429-4354-b51f-13f862921089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:34.020435 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f61e7ab-b429-4354-b51f-13f862921089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:34.020815 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f61e7ab-b429-4354-b51f-13f862921089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:34.021160 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f61e7ab-b429-4354-b51f-13f862921089 admin admin] PATH_INFO: `/v3/projects/f20e08e1248d44329bdb93e6e3f09796` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:34.022423 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f61e7ab-b429-4354-b51f-13f862921089 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f20e08e1248d44329bdb93e6e3f09796)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:34.024776 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f61e7ab-b429-4354-b51f-13f862921089 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:34.028041 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:34.047794 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859fec7a-c10b-48b4-950c-eb6e99d1b972 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:34.054764 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b66f6-5bcf-4388-89de-3affe91deac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:34.055437 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b66f6-5bcf-4388-89de-3affe91deac1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:34.056126 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b66f6-5bcf-4388-89de-3affe91deac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:34.056397 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b66f6-5bcf-4388-89de-3affe91deac1 admin admin] PATH_INFO: `/v3/users/7979f68c683b447aa4cf82470adb0eeb` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:34.058286 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b66f6-5bcf-4388-89de-3affe91deac1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7979f68c683b447aa4cf82470adb0eeb)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:34.060238 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b66f6-5bcf-4388-89de-3affe91deac1 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:34.190465 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-1f61e7ab-b429-4354-b51f-13f862921089 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f20e08e1248d44329bdb93e6e3f09796', 'request_id': 'req-1f61e7ab-b429-4354-b51f-13f862921089'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:34.197272 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-1f61e7ab-b429-4354-b51f-13f862921089 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f20e08e1248d44329bdb93e6e3f09796', 'request_id': 'req-1f61e7ab-b429-4354-b51f-13f862921089'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:34.248559 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-1f61e7ab-b429-4354-b51f-13f862921089 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f20e08e1248d44329bdb93e6e3f09796', 'request_id': 'req-1f61e7ab-b429-4354-b51f-13f862921089'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:34.260244 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-1f61e7ab-b429-4354-b51f-13f862921089 admin admin] The token cache is being invalidate because project f20e08e1248d44329bdb93e6e3f09796 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:51:34.260534 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-1f61e7ab-b429-4354-b51f-13f862921089 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:34.261171 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-1f61e7ab-b429-4354-b51f-13f862921089 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:34.262675 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1117/2218] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:51:33 2022] DELETE /identity/v3/projects/f20e08e1248d44329bdb93e6e3f09796 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:51:34.267694 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0cf1f3-3f43-4d1f-b7c6-2a965a0d32e9 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:34.267985 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0cf1f3-3f43-4d1f-b7c6-2a965a0d32e9 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:34.268264 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0cf1f3-3f43-4d1f-b7c6-2a965a0d32e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:34.291514 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-7a0cf1f3-3f43-4d1f-b7c6-2a965a0d32e9 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:34.292762 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-7a0cf1f3-3f43-4d1f-b7c6-2a965a0d32e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:34.307170 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-579b66f6-5bcf-4388-89de-3affe91deac1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7979f68c683b447aa4cf82470adb0eeb'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:34.318923 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-579b66f6-5bcf-4388-89de-3affe91deac1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7979f68c683b447aa4cf82470adb0eeb'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:34.319642 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-579b66f6-5bcf-4388-89de-3affe91deac1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7979f68c683b447aa4cf82470adb0eeb'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:34.326551 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-579b66f6-5bcf-4388-89de-3affe91deac1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7979f68c683b447aa4cf82470adb0eeb'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:34.355441 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1118/2219] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:34.364278 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42159714-e112-4e26-a59c-7992b84785ee None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:34.364690 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42159714-e112-4e26-a59c-7992b84785ee None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:34.367048 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42159714-e112-4e26-a59c-7992b84785ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:34.385140 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-42159714-e112-4e26-a59c-7992b84785ee None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:34.386350 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-42159714-e112-4e26-a59c-7992b84785ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:34.394253 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1119/2220] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:51:34.398531 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42159714-e112-4e26-a59c-7992b84785ee None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:34.404108 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:34.407693 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42159714-e112-4e26-a59c-7992b84785ee None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:34.413358 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa3ebf-23df-4248-b8fb-9d1297bc5389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:34.413955 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa3ebf-23df-4248-b8fb-9d1297bc5389 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:34.414224 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa3ebf-23df-4248-b8fb-9d1297bc5389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:34.414452 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-579b66f6-5bcf-4388-89de-3affe91deac1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7979f68c683b447aa4cf82470adb0eeb'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:34.414552 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa3ebf-23df-4248-b8fb-9d1297bc5389 admin admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:34.415110 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa3ebf-23df-4248-b8fb-9d1297bc5389 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:51:34.415439 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa3ebf-23df-4248-b8fb-9d1297bc5389 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:34.417010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa3ebf-23df-4248-b8fb-9d1297bc5389 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:34.421073 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1102/2221] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:51:33 2022] DELETE /identity/v3/users/7979f68c683b447aa4cf82470adb0eeb => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:51:34.431081 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1120/2222] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:51:34.435455 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b66f6-5bcf-4388-89de-3affe91deac1 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:34.504788 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b66f6-5bcf-4388-89de-3affe91deac1 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:34.509658 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f9b959-8cec-4d0a-a675-ae5ad1f665e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:34.510333 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9b959-8cec-4d0a-a675-ae5ad1f665e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:34.510615 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9b959-8cec-4d0a-a675-ae5ad1f665e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:34.510888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9b959-8cec-4d0a-a675-ae5ad1f665e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:34.511556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f9b959-8cec-4d0a-a675-ae5ad1f665e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:34.513460 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f9b959-8cec-4d0a-a675-ae5ad1f665e1 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:34.714384 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1103/2223] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:51:34 2022] POST /identity/v3/projects => generated 375 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:51:34.721189 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa3ebf-23df-4248-b8fb-9d1297bc5389 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:34.734059 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa3ebf-23df-4248-b8fb-9d1297bc5389 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:34.804758 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4f699e-b17c-402f-acca-ed3b89be41e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:34.805396 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f699e-b17c-402f-acca-ed3b89be41e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:34.805676 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f699e-b17c-402f-acca-ed3b89be41e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:34.805945 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f699e-b17c-402f-acca-ed3b89be41e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:34.806571 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4f699e-b17c-402f-acca-ed3b89be41e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:34.809054 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4f699e-b17c-402f-acca-ed3b89be41e9 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:34.928104 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1121/2224] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:51:34 2022] POST /identity/v3/users => generated 288 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:51:34.931805 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f9b959-8cec-4d0a-a675-ae5ad1f665e1 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:34.952036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f9b959-8cec-4d0a-a675-ae5ad1f665e1 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:34.955578 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd5001-cb54-4c0b-9d96-cccf61606ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:34.956196 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd5001-cb54-4c0b-9d96-cccf61606ac4 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:34.956470 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd5001-cb54-4c0b-9d96-cccf61606ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:34.956742 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd5001-cb54-4c0b-9d96-cccf61606ac4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:34.957970 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fd5001-cb54-4c0b-9d96-cccf61606ac4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:34.959885 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fd5001-cb54-4c0b-9d96-cccf61606ac4 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:34.972083 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1104/2225] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:51:34.975831 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4f699e-b17c-402f-acca-ed3b89be41e9 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:35.000198 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4f699e-b17c-402f-acca-ed3b89be41e9 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:35.007760 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaac634-331c-43de-97f5-dacad81dccad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:35.008384 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaac634-331c-43de-97f5-dacad81dccad admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:35.008660 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaac634-331c-43de-97f5-dacad81dccad admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:35.008927 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaac634-331c-43de-97f5-dacad81dccad admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:35.009566 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaac634-331c-43de-97f5-dacad81dccad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:35.011204 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaac634-331c-43de-97f5-dacad81dccad admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:35.026565 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1122/2226] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 12:51:35.031154 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd5001-cb54-4c0b-9d96-cccf61606ac4 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:35.042803 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd5001-cb54-4c0b-9d96-cccf61606ac4 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:35.049993 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bceefe-eec0-4bbe-bd6d-516657da7187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:35.049993 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bceefe-eec0-4bbe-bd6d-516657da7187 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:35.049993 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bceefe-eec0-4bbe-bd6d-516657da7187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:35.049993 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bceefe-eec0-4bbe-bd6d-516657da7187 admin admin] PATH_INFO: `/v3/projects/38d1778202d041b58934150c6af9685c/users/7ffffa27986f461ebc10c23c3c9817f0/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:35.069334 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bceefe-eec0-4bbe-bd6d-516657da7187 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38d1778202d041b58934150c6af9685c, user_id=7ffffa27986f461ebc10c23c3c9817f0, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:35.072660 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bceefe-eec0-4bbe-bd6d-516657da7187 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:35.115687 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaac634-331c-43de-97f5-dacad81dccad admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:35.117662 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-ebaac634-331c-43de-97f5-dacad81dccad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:35.136773 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaac634-331c-43de-97f5-dacad81dccad admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:35.153869 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf4a248-7dc2-4199-bb00-bbb58ada2826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:35.153869 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4a248-7dc2-4199-bb00-bbb58ada2826 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:35.153869 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4a248-7dc2-4199-bb00-bbb58ada2826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:35.153869 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4a248-7dc2-4199-bb00-bbb58ada2826 admin admin] PATH_INFO: `/v3/projects/63d62693a972457cac2c2834a93a9b23` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:35.154283 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf4a248-7dc2-4199-bb00-bbb58ada2826 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63d62693a972457cac2c2834a93a9b23)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:35.154882 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf4a248-7dc2-4199-bb00-bbb58ada2826 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:35.155397 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1105/2227] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:51:35 2022] PUT /identity/v3/projects/38d1778202d041b58934150c6af9685c/users/7ffffa27986f461ebc10c23c3c9817f0/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:51:35.161378 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24611123-e9a2-448e-ac3f-243b299754b4 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:35.162155 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24611123-e9a2-448e-ac3f-243b299754b4 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:35.162743 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24611123-e9a2-448e-ac3f-243b299754b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:35.190760 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-24611123-e9a2-448e-ac3f-243b299754b4 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:35.193124 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-24611123-e9a2-448e-ac3f-243b299754b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:35.206153 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:35.258588 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1106/2228] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:51:35.268248 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24611123-e9a2-448e-ac3f-243b299754b4 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:35.269754 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-24611123-e9a2-448e-ac3f-243b299754b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:35.306129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-edf4a248-7dc2-4199-bb00-bbb58ada2826 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63d62693a972457cac2c2834a93a9b23', 'request_id': 'req-edf4a248-7dc2-4199-bb00-bbb58ada2826'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:35.315097 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-edf4a248-7dc2-4199-bb00-bbb58ada2826 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63d62693a972457cac2c2834a93a9b23', 'request_id': 'req-edf4a248-7dc2-4199-bb00-bbb58ada2826'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:35.353592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24611123-e9a2-448e-ac3f-243b299754b4 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:35.369154 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-edf4a248-7dc2-4199-bb00-bbb58ada2826 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63d62693a972457cac2c2834a93a9b23', 'request_id': 'req-edf4a248-7dc2-4199-bb00-bbb58ada2826'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:35.386107 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-edf4a248-7dc2-4199-bb00-bbb58ada2826 admin admin] The token cache is being invalidate because project 63d62693a972457cac2c2834a93a9b23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:51:35.386107 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-edf4a248-7dc2-4199-bb00-bbb58ada2826 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:35.388422 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-edf4a248-7dc2-4199-bb00-bbb58ada2826 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:35.392304 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1123/2229] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:51:35 2022] DELETE /identity/v3/projects/63d62693a972457cac2c2834a93a9b23 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:35.397821 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a524c6-1a0f-46dd-b68e-3f3b3f1a9be9 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:35.398181 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a524c6-1a0f-46dd-b68e-3f3b3f1a9be9 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:35.398962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a524c6-1a0f-46dd-b68e-3f3b3f1a9be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:35.432683 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-a4a524c6-1a0f-46dd-b68e-3f3b3f1a9be9 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:35.433765 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-a4a524c6-1a0f-46dd-b68e-3f3b3f1a9be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:35.473547 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa08faa-b5fd-4887-8ff0-595961c63ca4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:35.474593 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa08faa-b5fd-4887-8ff0-595961c63ca4 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:35.474992 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa08faa-b5fd-4887-8ff0-595961c63ca4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:35.475398 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa08faa-b5fd-4887-8ff0-595961c63ca4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:35.477729 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-2fa08faa-b5fd-4887-8ff0-595961c63ca4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:35.515656 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:35.543706 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1124/2230] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:35.553172 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25aee3-265c-4b33-a907-dfcfb37e51c6 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:35.554201 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25aee3-265c-4b33-a907-dfcfb37e51c6 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:35.554891 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25aee3-265c-4b33-a907-dfcfb37e51c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:35.569367 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa08faa-b5fd-4887-8ff0-595961c63ca4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:35.570422 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa08faa-b5fd-4887-8ff0-595961c63ca4 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:35.583596 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1107/2231] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:51:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:51:35.586633 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-ee25aee3-265c-4b33-a907-dfcfb37e51c6 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:35.587901 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-ee25aee3-265c-4b33-a907-dfcfb37e51c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:35.596962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1125/2232] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:35.602962 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa08faa-b5fd-4887-8ff0-595961c63ca4 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:35.620987 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa08faa-b5fd-4887-8ff0-595961c63ca4 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:35.626703 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6593cf7-4311-4c53-9c2b-b65a238ef069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:35.627469 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6593cf7-4311-4c53-9c2b-b65a238ef069 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:35.629237 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6593cf7-4311-4c53-9c2b-b65a238ef069 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:35.629237 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6593cf7-4311-4c53-9c2b-b65a238ef069 admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:35.629237 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6593cf7-4311-4c53-9c2b-b65a238ef069 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:51:35.629237 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6593cf7-4311-4c53-9c2b-b65a238ef069 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:35.630942 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6593cf7-4311-4c53-9c2b-b65a238ef069 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:35.643326 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1108/2233] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:51:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:51:35.648033 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25aee3-265c-4b33-a907-dfcfb37e51c6 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:35.657215 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25aee3-265c-4b33-a907-dfcfb37e51c6 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:35.662725 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547de9b0-5ec4-4655-b8ad-98ecef9b2d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:35.663362 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547de9b0-5ec4-4655-b8ad-98ecef9b2d33 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:35.666080 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547de9b0-5ec4-4655-b8ad-98ecef9b2d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:35.666080 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547de9b0-5ec4-4655-b8ad-98ecef9b2d33 admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:35.666080 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547de9b0-5ec4-4655-b8ad-98ecef9b2d33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:35.666562 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547de9b0-5ec4-4655-b8ad-98ecef9b2d33 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:35.770474 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1126/2234] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:51:35 2022] POST /identity/v3/projects => generated 371 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:51:35.774448 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6593cf7-4311-4c53-9c2b-b65a238ef069 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:35.784521 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6593cf7-4311-4c53-9c2b-b65a238ef069 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:35.843349 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896707c-b25d-4876-970d-0e7e90b18f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:35.843955 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896707c-b25d-4876-970d-0e7e90b18f08 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:35.844239 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896707c-b25d-4876-970d-0e7e90b18f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:35.844509 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896707c-b25d-4876-970d-0e7e90b18f08 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:35.845280 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1896707c-b25d-4876-970d-0e7e90b18f08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:35.847210 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1896707c-b25d-4876-970d-0e7e90b18f08 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:36.012515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1109/2235] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:51:35 2022] POST /identity/v3/users => generated 286 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:51:36.013948 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547de9b0-5ec4-4655-b8ad-98ecef9b2d33 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:36.025385 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547de9b0-5ec4-4655-b8ad-98ecef9b2d33 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:36.029962 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c0fb4-a778-4ec4-9ce3-b26fcb39e47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:36.030592 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c0fb4-a778-4ec4-9ce3-b26fcb39e47d admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:36.030870 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c0fb4-a778-4ec4-9ce3-b26fcb39e47d admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:36.031138 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c0fb4-a778-4ec4-9ce3-b26fcb39e47d admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:36.031834 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2c0fb4-a778-4ec4-9ce3-b26fcb39e47d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:36.033429 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2c0fb4-a778-4ec4-9ce3-b26fcb39e47d admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:36.044942 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1127/2236] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:51:36.049662 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896707c-b25d-4876-970d-0e7e90b18f08 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:36.061240 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896707c-b25d-4876-970d-0e7e90b18f08 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:36.066145 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd59762-b885-46a7-ac36-3df9b5067d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:36.067094 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd59762-b885-46a7-ac36-3df9b5067d78 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:36.067415 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd59762-b885-46a7-ac36-3df9b5067d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:36.067691 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd59762-b885-46a7-ac36-3df9b5067d78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:36.070680 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd59762-b885-46a7-ac36-3df9b5067d78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:36.070680 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd59762-b885-46a7-ac36-3df9b5067d78 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:36.080770 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1110/2237] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:51:36.085539 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c0fb4-a778-4ec4-9ce3-b26fcb39e47d admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:36.096088 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c0fb4-a778-4ec4-9ce3-b26fcb39e47d admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:36.101619 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ba2b9-48fd-47f1-80cd-7fde4da4668b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:36.102444 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ba2b9-48fd-47f1-80cd-7fde4da4668b admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:36.103041 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ba2b9-48fd-47f1-80cd-7fde4da4668b admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:36.103041 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ba2b9-48fd-47f1-80cd-7fde4da4668b admin admin] PATH_INFO: `/v3/projects/6d115e48601640568b8a14b435fbdd2d/users/ac2c9d05e4474037b5032bbc4347594c/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:36.119486 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:36.121326 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0ba2b9-48fd-47f1-80cd-7fde4da4668b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d115e48601640568b8a14b435fbdd2d, user_id=ac2c9d05e4474037b5032bbc4347594c, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:36.124355 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0ba2b9-48fd-47f1-80cd-7fde4da4668b admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:36.249014 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1128/2238] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:51:36 2022] PUT /identity/v3/projects/6d115e48601640568b8a14b435fbdd2d/users/ac2c9d05e4474037b5032bbc4347594c/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:36.265673 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3863b-b014-4889-94bd-c683fc67ea86 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:36.265938 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3863b-b014-4889-94bd-c683fc67ea86 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:36.266210 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3863b-b014-4889-94bd-c683fc67ea86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:36.300871 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-dbb3863b-b014-4889-94bd-c683fc67ea86 None None] MFA Rules not processed for user `ac2c9d05e4474037b5032bbc4347594c`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:36.302231 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-dbb3863b-b014-4889-94bd-c683fc67ea86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:36.383048 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1111/2239] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:36.394240 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ba2b9-48fd-47f1-80cd-7fde4da4668b admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:36.396510 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-ac0ba2b9-48fd-47f1-80cd-7fde4da4668b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:36.461879 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ba2b9-48fd-47f1-80cd-7fde4da4668b admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:36.465691 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5726475f-34d8-439a-a154-59a00914ed16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:36.466243 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726475f-34d8-439a-a154-59a00914ed16 service cinder] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:36.466544 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726475f-34d8-439a-a154-59a00914ed16 service cinder] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:36.466816 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726475f-34d8-439a-a154-59a00914ed16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:36.476070 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5726475f-34d8-439a-a154-59a00914ed16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:36.477745 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5726475f-34d8-439a-a154-59a00914ed16 service cinder] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:36.486948 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1129/2240] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:51:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:51:36.498123 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb3863b-b014-4889-94bd-c683fc67ea86 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:36.499720 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-dbb3863b-b014-4889-94bd-c683fc67ea86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:36.509002 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5726475f-34d8-439a-a154-59a00914ed16 service cinder] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:36.588341 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb3863b-b014-4889-94bd-c683fc67ea86 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:36.594900 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5dcff5-690f-441f-aab7-a81fd908144c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:36.595997 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5dcff5-690f-441f-aab7-a81fd908144c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:36.596489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5dcff5-690f-441f-aab7-a81fd908144c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:36.596964 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5dcff5-690f-441f-aab7-a81fd908144c admin admin] PATH_INFO: `/v3/users/9f831ee08c654a5b9fd63598f3fa8c9e` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:36.599059 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5dcff5-690f-441f-aab7-a81fd908144c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f831ee08c654a5b9fd63598f3fa8c9e)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:36.601098 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5dcff5-690f-441f-aab7-a81fd908144c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:36.601538 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5726475f-34d8-439a-a154-59a00914ed16 service cinder] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:36.606697 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedf811e-df31-46e2-ab52-e13225fa1e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:36.607933 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf811e-df31-46e2-ab52-e13225fa1e59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:36.607933 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf811e-df31-46e2-ab52-e13225fa1e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:36.608403 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf811e-df31-46e2-ab52-e13225fa1e59 admin admin] PATH_INFO: `/v3/users/ac2c9d05e4474037b5032bbc4347594c` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:36.609817 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedf811e-df31-46e2-ab52-e13225fa1e59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac2c9d05e4474037b5032bbc4347594c)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:36.611692 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedf811e-df31-46e2-ab52-e13225fa1e59 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:36.628284 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:36.803817 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:36.818977 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-eedf811e-df31-46e2-ab52-e13225fa1e59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac2c9d05e4474037b5032bbc4347594c'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:36.828213 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-eedf811e-df31-46e2-ab52-e13225fa1e59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac2c9d05e4474037b5032bbc4347594c'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:36.841140 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-3b5dcff5-690f-441f-aab7-a81fd908144c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f831ee08c654a5b9fd63598f3fa8c9e'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:36.844773 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-eedf811e-df31-46e2-ab52-e13225fa1e59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac2c9d05e4474037b5032bbc4347594c'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:36.845594 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-eedf811e-df31-46e2-ab52-e13225fa1e59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac2c9d05e4474037b5032bbc4347594c'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:36.854704 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-3b5dcff5-690f-441f-aab7-a81fd908144c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f831ee08c654a5b9fd63598f3fa8c9e'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:36.855790 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-3b5dcff5-690f-441f-aab7-a81fd908144c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f831ee08c654a5b9fd63598f3fa8c9e'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:36.866902 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-3b5dcff5-690f-441f-aab7-a81fd908144c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f831ee08c654a5b9fd63598f3fa8c9e'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:36.939861 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-eedf811e-df31-46e2-ab52-e13225fa1e59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac2c9d05e4474037b5032bbc4347594c'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:36.947293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1130/2241] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:51:36 2022] DELETE /identity/v3/users/ac2c9d05e4474037b5032bbc4347594c => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:51:36.951542 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ba9eb-df60-4ae3-86e5-9c3f9640d74d None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:36.951833 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ba9eb-df60-4ae3-86e5-9c3f9640d74d None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:36.952270 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ba9eb-df60-4ae3-86e5-9c3f9640d74d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:36.972364 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-c48ba9eb-df60-4ae3-86e5-9c3f9640d74d None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:36.974174 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-c48ba9eb-df60-4ae3-86e5-9c3f9640d74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:37.014136 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-3b5dcff5-690f-441f-aab7-a81fd908144c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f831ee08c654a5b9fd63598f3fa8c9e'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:37.020654 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1112/2242] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:51:36 2022] DELETE /identity/v3/users/9f831ee08c654a5b9fd63598f3fa8c9e => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:37.025235 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3747273-ef3c-4fa1-a5c1-df909388572e None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:37.025602 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3747273-ef3c-4fa1-a5c1-df909388572e None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:37.025873 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3747273-ef3c-4fa1-a5c1-df909388572e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:37.039870 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1131/2243] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:37.046184 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-a3747273-ef3c-4fa1-a5c1-df909388572e None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:37.047408 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a3747273-ef3c-4fa1-a5c1-df909388572e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:37.057547 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ba9eb-df60-4ae3-86e5-9c3f9640d74d None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:37.058886 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-c48ba9eb-df60-4ae3-86e5-9c3f9640d74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:37.108253 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1113/2244] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:37.117027 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ba9eb-df60-4ae3-86e5-9c3f9640d74d None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:37.124674 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3747273-ef3c-4fa1-a5c1-df909388572e None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:37.126209 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a3747273-ef3c-4fa1-a5c1-df909388572e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:37.175614 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a39469-5a91-4575-a31f-2c3b7a617859 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:37.177184 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a39469-5a91-4575-a31f-2c3b7a617859 service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:37.177658 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a39469-5a91-4575-a31f-2c3b7a617859 service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:37.177956 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a39469-5a91-4575-a31f-2c3b7a617859 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:37.187862 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a39469-5a91-4575-a31f-2c3b7a617859 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:37.189284 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a39469-5a91-4575-a31f-2c3b7a617859 service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:37.193675 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3747273-ef3c-4fa1-a5c1-df909388572e None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:37.198023 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d587f-00b2-41d9-bc28-7339b9043e01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:37.198833 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d587f-00b2-41d9-bc28-7339b9043e01 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:37.199528 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d587f-00b2-41d9-bc28-7339b9043e01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:37.199947 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1132/2245] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:51:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:51:37.200236 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d587f-00b2-41d9-bc28-7339b9043e01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:37.212032 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257d587f-00b2-41d9-bc28-7339b9043e01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:37.213900 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257d587f-00b2-41d9-bc28-7339b9043e01 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:37.227439 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1114/2246] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:51:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:51:37.839834 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a39469-5a91-4575-a31f-2c3b7a617859 service neutron] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:37.842373 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-b0a39469-5a91-4575-a31f-2c3b7a617859 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:37.864863 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a39469-5a91-4575-a31f-2c3b7a617859 service neutron] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:37.871270 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e79fcb-a702-43f5-b025-ad75d98e232b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:37.872945 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e79fcb-a702-43f5-b025-ad75d98e232b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:37.872945 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e79fcb-a702-43f5-b025-ad75d98e232b admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:37.872945 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e79fcb-a702-43f5-b025-ad75d98e232b admin admin] PATH_INFO: `/v3/projects/55d56aa662684aef99e02942c56b3e9e` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:37.877923 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e79fcb-a702-43f5-b025-ad75d98e232b admin admin] RBAC: Authorizing `identity:delete_project(project_id=55d56aa662684aef99e02942c56b3e9e)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:37.878003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e79fcb-a702-43f5-b025-ad75d98e232b admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:37.878773 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d587f-00b2-41d9-bc28-7339b9043e01 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:37.880555 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-257d587f-00b2-41d9-bc28-7339b9043e01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:37.912096 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d587f-00b2-41d9-bc28-7339b9043e01 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:37.931091 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5c6a3d-d138-412f-b864-7db41a2f7378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:37.931822 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c6a3d-d138-412f-b864-7db41a2f7378 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:37.932100 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c6a3d-d138-412f-b864-7db41a2f7378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:37.932376 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c6a3d-d138-412f-b864-7db41a2f7378 admin admin] PATH_INFO: `/v3/projects/6d115e48601640568b8a14b435fbdd2d` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:37.936098 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5c6a3d-d138-412f-b864-7db41a2f7378 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d115e48601640568b8a14b435fbdd2d)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:37.940672 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5c6a3d-d138-412f-b864-7db41a2f7378 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:38.384733 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-69e79fcb-a702-43f5-b025-ad75d98e232b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55d56aa662684aef99e02942c56b3e9e', 'request_id': 'req-69e79fcb-a702-43f5-b025-ad75d98e232b'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:38.385400 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9f5c6a3d-d138-412f-b864-7db41a2f7378 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d115e48601640568b8a14b435fbdd2d', 'request_id': 'req-9f5c6a3d-d138-412f-b864-7db41a2f7378'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:38.392736 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9f5c6a3d-d138-412f-b864-7db41a2f7378 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d115e48601640568b8a14b435fbdd2d', 'request_id': 'req-9f5c6a3d-d138-412f-b864-7db41a2f7378'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:38.398514 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-69e79fcb-a702-43f5-b025-ad75d98e232b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55d56aa662684aef99e02942c56b3e9e', 'request_id': 'req-69e79fcb-a702-43f5-b025-ad75d98e232b'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:38.535442 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9f5c6a3d-d138-412f-b864-7db41a2f7378 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d115e48601640568b8a14b435fbdd2d', 'request_id': 'req-9f5c6a3d-d138-412f-b864-7db41a2f7378'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:38.550219 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9f5c6a3d-d138-412f-b864-7db41a2f7378 admin admin] The token cache is being invalidate because project 6d115e48601640568b8a14b435fbdd2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:51:38.550658 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9f5c6a3d-d138-412f-b864-7db41a2f7378 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:38.552199 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9f5c6a3d-d138-412f-b864-7db41a2f7378 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:38.557307 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1115/2247] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:51:37 2022] DELETE /identity/v3/projects/6d115e48601640568b8a14b435fbdd2d => generated 0 bytes in 676 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:38.558805 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f15217-a6ce-46af-832d-b5bb833512f1 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:38.559093 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f15217-a6ce-46af-832d-b5bb833512f1 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:38.559419 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f15217-a6ce-46af-832d-b5bb833512f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:38.592065 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-20f15217-a6ce-46af-832d-b5bb833512f1 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:38.593293 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-20f15217-a6ce-46af-832d-b5bb833512f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:38.650789 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-69e79fcb-a702-43f5-b025-ad75d98e232b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55d56aa662684aef99e02942c56b3e9e', 'request_id': 'req-69e79fcb-a702-43f5-b025-ad75d98e232b'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:38.672788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-69e79fcb-a702-43f5-b025-ad75d98e232b admin admin] The token cache is being invalidate because project 55d56aa662684aef99e02942c56b3e9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:51:38.673080 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-69e79fcb-a702-43f5-b025-ad75d98e232b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:38.676905 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-69e79fcb-a702-43f5-b025-ad75d98e232b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:38.679718 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1133/2248] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:51:37 2022] DELETE /identity/v3/projects/55d56aa662684aef99e02942c56b3e9e => generated 0 bytes in 841 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:38.737611 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1116/2249] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:51:38.748547 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735c18b-644a-4ce8-afb8-0066e622cba3 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:38.748945 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735c18b-644a-4ce8-afb8-0066e622cba3 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:38.749288 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735c18b-644a-4ce8-afb8-0066e622cba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:38.781203 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-0735c18b-644a-4ce8-afb8-0066e622cba3 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:38.783016 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-0735c18b-644a-4ce8-afb8-0066e622cba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:38.797534 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1134/2250] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:38.807169 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f15217-a6ce-46af-832d-b5bb833512f1 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:38.829058 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f15217-a6ce-46af-832d-b5bb833512f1 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:38.843237 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ccccae-0cb1-4e73-a194-bce5fc86016b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:38.844149 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ccccae-0cb1-4e73-a194-bce5fc86016b admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:38.844511 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ccccae-0cb1-4e73-a194-bce5fc86016b admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:38.845235 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ccccae-0cb1-4e73-a194-bce5fc86016b admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:38.845620 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ccccae-0cb1-4e73-a194-bce5fc86016b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:51:38.847024 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ccccae-0cb1-4e73-a194-bce5fc86016b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:38.851697 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ccccae-0cb1-4e73-a194-bce5fc86016b admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:38.866881 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:38.877507 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1117/2251] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:51:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:51:38.880801 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735c18b-644a-4ce8-afb8-0066e622cba3 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:38.898921 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735c18b-644a-4ce8-afb8-0066e622cba3 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:38.908365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f921d6de-60ed-4177-9cee-94271de4cbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:38.909259 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921d6de-60ed-4177-9cee-94271de4cbfb admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:38.909543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921d6de-60ed-4177-9cee-94271de4cbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:38.910048 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921d6de-60ed-4177-9cee-94271de4cbfb admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:38.911001 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f921d6de-60ed-4177-9cee-94271de4cbfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:38.913190 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f921d6de-60ed-4177-9cee-94271de4cbfb admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:39.035384 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1135/2252] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:51:38 2022] POST /identity/v3/projects => generated 385 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:51:39.037857 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ccccae-0cb1-4e73-a194-bce5fc86016b admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:39.052558 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ccccae-0cb1-4e73-a194-bce5fc86016b admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:39.149106 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94563cf8-bdbb-407a-b1d7-e101d2ddbb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:39.150340 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94563cf8-bdbb-407a-b1d7-e101d2ddbb10 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:39.150786 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94563cf8-bdbb-407a-b1d7-e101d2ddbb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:39.154802 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94563cf8-bdbb-407a-b1d7-e101d2ddbb10 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:39.154802 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94563cf8-bdbb-407a-b1d7-e101d2ddbb10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:39.155530 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94563cf8-bdbb-407a-b1d7-e101d2ddbb10 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:39.296219 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1118/2253] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:51:39 2022] POST /identity/v3/users => generated 293 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:51:39.304636 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f921d6de-60ed-4177-9cee-94271de4cbfb admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:39.320803 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:39.326165 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f921d6de-60ed-4177-9cee-94271de4cbfb admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:39.340819 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc45944-de31-4723-bfca-b010898e3201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:39.341440 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc45944-de31-4723-bfca-b010898e3201 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:39.341723 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc45944-de31-4723-bfca-b010898e3201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:39.341997 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc45944-de31-4723-bfca-b010898e3201 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:39.342650 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc45944-de31-4723-bfca-b010898e3201 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:39.344558 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc45944-de31-4723-bfca-b010898e3201 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:39.364240 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94563cf8-bdbb-407a-b1d7-e101d2ddbb10 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:39.368043 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1136/2254] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:51:39.388382 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94563cf8-bdbb-407a-b1d7-e101d2ddbb10 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:39.400166 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f33ee3-c7ef-4160-b327-9c9f4d067bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:39.401137 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f33ee3-c7ef-4160-b327-9c9f4d067bb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:39.401587 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f33ee3-c7ef-4160-b327-9c9f4d067bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:39.402015 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f33ee3-c7ef-4160-b327-9c9f4d067bb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:39.402870 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f33ee3-c7ef-4160-b327-9c9f4d067bb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:39.406023 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f33ee3-c7ef-4160-b327-9c9f4d067bb2 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:39.426762 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1119/2255] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:51:39.431674 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc45944-de31-4723-bfca-b010898e3201 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:39.446302 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc45944-de31-4723-bfca-b010898e3201 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:39.452998 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f2e7e-203c-4c92-ba20-63e6392ad1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:39.454056 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f2e7e-203c-4c92-ba20-63e6392ad1ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:39.454346 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f2e7e-203c-4c92-ba20-63e6392ad1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:39.454617 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f2e7e-203c-4c92-ba20-63e6392ad1ec admin admin] PATH_INFO: `/v3/projects/01d9d29160e54fe79fa64f0b70f0a536/users/2fdfdc6002dd42ca8471dcbc608fdfbd/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:39.470907 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4f2e7e-203c-4c92-ba20-63e6392ad1ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=01d9d29160e54fe79fa64f0b70f0a536, user_id=2fdfdc6002dd42ca8471dcbc608fdfbd, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:39.474509 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4f2e7e-203c-4c92-ba20-63e6392ad1ec admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:39.562910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f33ee3-c7ef-4160-b327-9c9f4d067bb2 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:39.565533 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-60f33ee3-c7ef-4160-b327-9c9f4d067bb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:39.584721 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1137/2256] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:51:39 2022] PUT /identity/v3/projects/01d9d29160e54fe79fa64f0b70f0a536/users/2fdfdc6002dd42ca8471dcbc608fdfbd/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:39.586664 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f33ee3-c7ef-4160-b327-9c9f4d067bb2 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:39.597351 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a68f7-cb4b-47c7-82de-4f3400b87fba None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:39.598312 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a68f7-cb4b-47c7-82de-4f3400b87fba None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:39.598799 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a68f7-cb4b-47c7-82de-4f3400b87fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:39.640302 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-b59a68f7-cb4b-47c7-82de-4f3400b87fba None None] MFA Rules not processed for user `2fdfdc6002dd42ca8471dcbc608fdfbd`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:39.641601 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-b59a68f7-cb4b-47c7-82de-4f3400b87fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:39.673980 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae53250-ec33-48f3-9910-166acd9db1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:39.675782 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae53250-ec33-48f3-9910-166acd9db1d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:39.675782 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae53250-ec33-48f3-9910-166acd9db1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:39.676982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae53250-ec33-48f3-9910-166acd9db1d0 admin admin] PATH_INFO: `/v3/users/090335bc919c4d3b873c6d482e464033` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:39.678288 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae53250-ec33-48f3-9910-166acd9db1d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=090335bc919c4d3b873c6d482e464033)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:39.682032 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae53250-ec33-48f3-9910-166acd9db1d0 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:39.739861 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1138/2257] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:39.750272 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a68f7-cb4b-47c7-82de-4f3400b87fba None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:39.764405 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a68f7-cb4b-47c7-82de-4f3400b87fba None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:39.770471 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b4bf5-ddb8-4990-9614-d213fce30731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:39.771215 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b4bf5-ddb8-4990-9614-d213fce30731 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:39.771532 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b4bf5-ddb8-4990-9614-d213fce30731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:39.771807 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b4bf5-ddb8-4990-9614-d213fce30731 admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:39.772454 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108b4bf5-ddb8-4990-9614-d213fce30731 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:39.774438 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108b4bf5-ddb8-4990-9614-d213fce30731 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:39.880657 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1139/2258] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:51:39 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:51:39.885001 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b4bf5-ddb8-4990-9614-d213fce30731 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:39.916172 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4ae53250-ec33-48f3-9910-166acd9db1d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '090335bc919c4d3b873c6d482e464033'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:39.932019 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4ae53250-ec33-48f3-9910-166acd9db1d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '090335bc919c4d3b873c6d482e464033'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:39.932889 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4ae53250-ec33-48f3-9910-166acd9db1d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '090335bc919c4d3b873c6d482e464033'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:39.941060 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4ae53250-ec33-48f3-9910-166acd9db1d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '090335bc919c4d3b873c6d482e464033'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:39.962361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b4bf5-ddb8-4990-9614-d213fce30731 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:39.969293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddf479d-3038-475e-a643-93ef2b0e23bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:39.969928 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf479d-3038-475e-a643-93ef2b0e23bd admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:39.970215 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf479d-3038-475e-a643-93ef2b0e23bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:39.970493 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf479d-3038-475e-a643-93ef2b0e23bd admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:39.971131 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddf479d-3038-475e-a643-93ef2b0e23bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:39.975685 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddf479d-3038-475e-a643-93ef2b0e23bd admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:40.031556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4ae53250-ec33-48f3-9910-166acd9db1d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '090335bc919c4d3b873c6d482e464033'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:40.040367 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1120/2259] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:51:39 2022] DELETE /identity/v3/users/090335bc919c4d3b873c6d482e464033 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:51:40.044737 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f2d78-13f2-4645-8e7a-fea3eb223d8b None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:40.045118 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f2d78-13f2-4645-8e7a-fea3eb223d8b None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:40.045702 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f2d78-13f2-4645-8e7a-fea3eb223d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:40.099417 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-216f2d78-13f2-4645-8e7a-fea3eb223d8b None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:40.100799 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-216f2d78-13f2-4645-8e7a-fea3eb223d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:40.148931 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1140/2260] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:51:39 2022] POST /identity/v3/users => generated 293 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:51:40.154287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddf479d-3038-475e-a643-93ef2b0e23bd admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:40.154287 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-dddf479d-3038-475e-a643-93ef2b0e23bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:40.214151 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddf479d-3038-475e-a643-93ef2b0e23bd admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:40.223351 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1121/2261] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:40.226016 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d3490a-bf94-40d0-b170-b6af3370794b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:40.228557 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d3490a-bf94-40d0-b170-b6af3370794b admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:40.229005 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d3490a-bf94-40d0-b170-b6af3370794b admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:40.229290 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d3490a-bf94-40d0-b170-b6af3370794b admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:40.234565 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d3490a-bf94-40d0-b170-b6af3370794b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:40.237035 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d3490a-bf94-40d0-b170-b6af3370794b admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:40.241100 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f2d78-13f2-4645-8e7a-fea3eb223d8b None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:40.242802 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-216f2d78-13f2-4645-8e7a-fea3eb223d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:40.266504 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1141/2262] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:40 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:51:40.272478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d3490a-bf94-40d0-b170-b6af3370794b admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:40.283184 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d3490a-bf94-40d0-b170-b6af3370794b admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:40.289718 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c6060-72b3-4e40-8bfe-0c9ce7bc0134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:40.293053 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c6060-72b3-4e40-8bfe-0c9ce7bc0134 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:40.293053 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c6060-72b3-4e40-8bfe-0c9ce7bc0134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:40.293053 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c6060-72b3-4e40-8bfe-0c9ce7bc0134 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:40.293053 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1c6060-72b3-4e40-8bfe-0c9ce7bc0134 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:40.294019 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1c6060-72b3-4e40-8bfe-0c9ce7bc0134 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:40.326877 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1142/2263] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:40 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:51:40.326877 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c6060-72b3-4e40-8bfe-0c9ce7bc0134 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:40.349259 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c6060-72b3-4e40-8bfe-0c9ce7bc0134 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:40.362584 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b91974e-7cf8-4a73-911a-9ac67638a4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:40.364208 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b91974e-7cf8-4a73-911a-9ac67638a4c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:40.364340 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b91974e-7cf8-4a73-911a-9ac67638a4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:40.364607 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b91974e-7cf8-4a73-911a-9ac67638a4c1 admin admin] PATH_INFO: `/v3/projects/65cd267ba9854f5ab7e729ab115318d1/users/9f6ef09c696948e59075dd110bb5904f/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:40.377140 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f2d78-13f2-4645-8e7a-fea3eb223d8b None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:40.382618 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a59e9f-8afc-4e75-914a-04e6924d7fb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:40.384470 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a59e9f-8afc-4e75-914a-04e6924d7fb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:40.384470 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a59e9f-8afc-4e75-914a-04e6924d7fb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:40.384470 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a59e9f-8afc-4e75-914a-04e6924d7fb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:40.386509 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b91974e-7cf8-4a73-911a-9ac67638a4c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65cd267ba9854f5ab7e729ab115318d1, user_id=9f6ef09c696948e59075dd110bb5904f, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:40.390119 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b91974e-7cf8-4a73-911a-9ac67638a4c1 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:40.394813 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a59e9f-8afc-4e75-914a-04e6924d7fb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:40.396315 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a59e9f-8afc-4e75-914a-04e6924d7fb5 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:40.407166 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1122/2264] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:51:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:51:40.486847 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1143/2265] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:51:40 2022] PUT /identity/v3/projects/65cd267ba9854f5ab7e729ab115318d1/users/9f6ef09c696948e59075dd110bb5904f/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:40.498310 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aebab4-3830-4a22-8bdb-1ce84b481557 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:40.498592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aebab4-3830-4a22-8bdb-1ce84b481557 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:40.498869 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aebab4-3830-4a22-8bdb-1ce84b481557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:40.538663 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-67aebab4-3830-4a22-8bdb-1ce84b481557 None None] MFA Rules not processed for user `9f6ef09c696948e59075dd110bb5904f`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:40.540584 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-67aebab4-3830-4a22-8bdb-1ce84b481557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:40.626844 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1123/2266] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:40.633905 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b91974e-7cf8-4a73-911a-9ac67638a4c1 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:40.719945 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b91974e-7cf8-4a73-911a-9ac67638a4c1 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:40.726148 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77700253-c63d-4ba6-9ab6-01332d20a131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:40.726776 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77700253-c63d-4ba6-9ab6-01332d20a131 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:40.727058 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77700253-c63d-4ba6-9ab6-01332d20a131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:40.727343 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77700253-c63d-4ba6-9ab6-01332d20a131 admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:40.728124 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77700253-c63d-4ba6-9ab6-01332d20a131 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:40.730118 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77700253-c63d-4ba6-9ab6-01332d20a131 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:40.858352 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1144/2267] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:51:40 2022] POST /identity/v3/projects => generated 383 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:51:40.862849 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aebab4-3830-4a22-8bdb-1ce84b481557 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:40.895849 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aebab4-3830-4a22-8bdb-1ce84b481557 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:40.936629 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:41.001863 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e574d9-ab56-4e6b-84ab-8357288efd3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:41.002652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e574d9-ab56-4e6b-84ab-8357288efd3e admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:41.003011 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e574d9-ab56-4e6b-84ab-8357288efd3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:41.003455 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e574d9-ab56-4e6b-84ab-8357288efd3e admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:41.007350 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e574d9-ab56-4e6b-84ab-8357288efd3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:41.007524 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e574d9-ab56-4e6b-84ab-8357288efd3e admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:41.119211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77700253-c63d-4ba6-9ab6-01332d20a131 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:41.123168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-77700253-c63d-4ba6-9ab6-01332d20a131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:41.131600 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1124/2268] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:51:40 2022] POST /identity/v3/users => generated 298 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:51:41.141119 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e574d9-ab56-4e6b-84ab-8357288efd3e admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:41.155493 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77700253-c63d-4ba6-9ab6-01332d20a131 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:41.155690 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e574d9-ab56-4e6b-84ab-8357288efd3e admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:41.161784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f511063-e4f6-452d-943d-44d06140da13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:41.167216 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f511063-e4f6-452d-943d-44d06140da13 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:41.167216 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f511063-e4f6-452d-943d-44d06140da13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:41.167216 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f511063-e4f6-452d-943d-44d06140da13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:41.167216 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f511063-e4f6-452d-943d-44d06140da13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:41.167216 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f511063-e4f6-452d-943d-44d06140da13 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:41.174808 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ec17f-0a95-44aa-b850-381b8e741e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:41.175620 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ec17f-0a95-44aa-b850-381b8e741e42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:41.176992 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ec17f-0a95-44aa-b850-381b8e741e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:41.176992 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ec17f-0a95-44aa-b850-381b8e741e42 admin admin] PATH_INFO: `/v3/projects/b22cd583306248a1b2f052414af818f1` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:41.179520 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ec17f-0a95-44aa-b850-381b8e741e42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b22cd583306248a1b2f052414af818f1)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:41.183103 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1125/2269] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:51:41.188725 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ec17f-0a95-44aa-b850-381b8e741e42 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:41.188982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f511063-e4f6-452d-943d-44d06140da13 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:41.205404 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f511063-e4f6-452d-943d-44d06140da13 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:41.212446 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15da51-6d08-42a2-b33b-feee64ca74fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:41.213077 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15da51-6d08-42a2-b33b-feee64ca74fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:41.213363 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15da51-6d08-42a2-b33b-feee64ca74fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:41.213639 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15da51-6d08-42a2-b33b-feee64ca74fa admin admin] PATH_INFO: `/v3/projects/d6bc5767b2934d85bc754a037a5c97d6/users/dba7c84c8bed4568802bbaf15f6c2c24/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:41.233138 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c15da51-6d08-42a2-b33b-feee64ca74fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6bc5767b2934d85bc754a037a5c97d6, user_id=dba7c84c8bed4568802bbaf15f6c2c24, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:41.249331 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c15da51-6d08-42a2-b33b-feee64ca74fa admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:41.342344 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1126/2270] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:51:41 2022] PUT /identity/v3/projects/d6bc5767b2934d85bc754a037a5c97d6/users/dba7c84c8bed4568802bbaf15f6c2c24/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:41.346086 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15da51-6d08-42a2-b33b-feee64ca74fa admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:41.355123 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-c01ec17f-0a95-44aa-b850-381b8e741e42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b22cd583306248a1b2f052414af818f1', 'request_id': 'req-c01ec17f-0a95-44aa-b850-381b8e741e42'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:41.374070 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-c01ec17f-0a95-44aa-b850-381b8e741e42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b22cd583306248a1b2f052414af818f1', 'request_id': 'req-c01ec17f-0a95-44aa-b850-381b8e741e42'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:41.434266 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15da51-6d08-42a2-b33b-feee64ca74fa admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:41.444060 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-c01ec17f-0a95-44aa-b850-381b8e741e42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b22cd583306248a1b2f052414af818f1', 'request_id': 'req-c01ec17f-0a95-44aa-b850-381b8e741e42'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:41.458175 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-c01ec17f-0a95-44aa-b850-381b8e741e42 admin admin] The token cache is being invalidate because project b22cd583306248a1b2f052414af818f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:51:41.458505 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-c01ec17f-0a95-44aa-b850-381b8e741e42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:41.459119 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-c01ec17f-0a95-44aa-b850-381b8e741e42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:41.460665 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1145/2271] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:51:41 2022] DELETE /identity/v3/projects/b22cd583306248a1b2f052414af818f1 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:41.464621 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ec17f-0a95-44aa-b850-381b8e741e42 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:41.466208 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-c01ec17f-0a95-44aa-b850-381b8e741e42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:41.522988 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0767ba09-a914-4d07-ba58-ee7c269fc867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:41.523844 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0767ba09-a914-4d07-ba58-ee7c269fc867 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:41.524205 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0767ba09-a914-4d07-ba58-ee7c269fc867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:41.524560 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0767ba09-a914-4d07-ba58-ee7c269fc867 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:41.525284 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0767ba09-a914-4d07-ba58-ee7c269fc867 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:41.526990 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0767ba09-a914-4d07-ba58-ee7c269fc867 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:41.533399 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:41.556083 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1127/2272] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:51:41.558949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0767ba09-a914-4d07-ba58-ee7c269fc867 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:41.562263 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-0767ba09-a914-4d07-ba58-ee7c269fc867 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:41.576365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ec17f-0a95-44aa-b850-381b8e741e42 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:41.578259 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0767ba09-a914-4d07-ba58-ee7c269fc867 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:41.582975 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bd268f-0589-400d-998a-241c55305864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:41.583793 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c39a9-9767-4c09-a166-740bfe2cfa7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:41.584294 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bd268f-0589-400d-998a-241c55305864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:41.584675 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c39a9-9767-4c09-a166-740bfe2cfa7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:41.584895 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bd268f-0589-400d-998a-241c55305864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:41.585080 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c39a9-9767-4c09-a166-740bfe2cfa7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:41.585420 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c39a9-9767-4c09-a166-740bfe2cfa7f admin admin] PATH_INFO: `/v3/projects/d6bc5767b2934d85bc754a037a5c97d6/users/dba7c84c8bed4568802bbaf15f6c2c24/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:41.585634 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bd268f-0589-400d-998a-241c55305864 admin admin] PATH_INFO: `/v3/users/96fe6b68eaf14e8bab52f257ad7b556c` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:41.587265 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bd268f-0589-400d-998a-241c55305864 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96fe6b68eaf14e8bab52f257ad7b556c)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:41.587828 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c39a9-9767-4c09-a166-740bfe2cfa7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6bc5767b2934d85bc754a037a5c97d6, user_id=dba7c84c8bed4568802bbaf15f6c2c24, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:41.589678 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bd268f-0589-400d-998a-241c55305864 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:41.590974 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c39a9-9767-4c09-a166-740bfe2cfa7f admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:41.609919 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1128/2273] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:51:41 2022] PUT /identity/v3/projects/d6bc5767b2934d85bc754a037a5c97d6/users/dba7c84c8bed4568802bbaf15f6c2c24/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:41.624837 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c495c-db9c-45a6-9a1a-a19dea74984e None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:41.629631 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c495c-db9c-45a6-9a1a-a19dea74984e None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:41.629631 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c495c-db9c-45a6-9a1a-a19dea74984e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:41.684876 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-405c495c-db9c-45a6-9a1a-a19dea74984e None None] MFA Rules not processed for user `dba7c84c8bed4568802bbaf15f6c2c24`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:41.686130 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-405c495c-db9c-45a6-9a1a-a19dea74984e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:41.812537 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1129/2274] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:41 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:51:41.817201 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-e5bd268f-0589-400d-998a-241c55305864 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96fe6b68eaf14e8bab52f257ad7b556c'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:41.825283 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c495c-db9c-45a6-9a1a-a19dea74984e None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:41.831499 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-405c495c-db9c-45a6-9a1a-a19dea74984e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:41.840911 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-e5bd268f-0589-400d-998a-241c55305864 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96fe6b68eaf14e8bab52f257ad7b556c'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:41.870393 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-e5bd268f-0589-400d-998a-241c55305864 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96fe6b68eaf14e8bab52f257ad7b556c'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:41.871789 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-e5bd268f-0589-400d-998a-241c55305864 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96fe6b68eaf14e8bab52f257ad7b556c'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:41.947894 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c495c-db9c-45a6-9a1a-a19dea74984e None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:41.952108 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dab179-c668-4ba7-92bd-7db223463a45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:41.952734 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dab179-c668-4ba7-92bd-7db223463a45 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:41.956757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dab179-c668-4ba7-92bd-7db223463a45 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:41.956757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dab179-c668-4ba7-92bd-7db223463a45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:41.956757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e1dab179-c668-4ba7-92bd-7db223463a45 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:41.963201 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-e5bd268f-0589-400d-998a-241c55305864 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96fe6b68eaf14e8bab52f257ad7b556c'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:41.982624 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1146/2275] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:51:41 2022] DELETE /identity/v3/users/96fe6b68eaf14e8bab52f257ad7b556c => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:51:42.055049 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dab179-c668-4ba7-92bd-7db223463a45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:42.056656 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dab179-c668-4ba7-92bd-7db223463a45 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:42.081360 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1130/2276] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:51:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:51:43.084886 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bd268f-0589-400d-998a-241c55305864 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:43.086490 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-e5bd268f-0589-400d-998a-241c55305864 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:43.255173 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bd268f-0589-400d-998a-241c55305864 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:43.261856 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891dcb3c-8718-4a16-a2a4-49f3970dbcd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:43.262851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891dcb3c-8718-4a16-a2a4-49f3970dbcd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:43.263841 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891dcb3c-8718-4a16-a2a4-49f3970dbcd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:43.264328 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891dcb3c-8718-4a16-a2a4-49f3970dbcd0 admin admin] PATH_INFO: `/v3/projects/fd710dd7ca454d819aaf50dc7fc7372f` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:43.266197 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891dcb3c-8718-4a16-a2a4-49f3970dbcd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd710dd7ca454d819aaf50dc7fc7372f)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:43.268745 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891dcb3c-8718-4a16-a2a4-49f3970dbcd0 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:43.284907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf939e55-40cc-4943-981c-2b59dd1463a6 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:43.285882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf939e55-40cc-4943-981c-2b59dd1463a6 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:43.285882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf939e55-40cc-4943-981c-2b59dd1463a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:43.350608 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-cf939e55-40cc-4943-981c-2b59dd1463a6 None None] MFA Rules not processed for user `7ffffa27986f461ebc10c23c3c9817f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:43.352031 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-cf939e55-40cc-4943-981c-2b59dd1463a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:43.431148 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-891dcb3c-8718-4a16-a2a4-49f3970dbcd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd710dd7ca454d819aaf50dc7fc7372f', 'request_id': 'req-891dcb3c-8718-4a16-a2a4-49f3970dbcd0'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:43.445635 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-891dcb3c-8718-4a16-a2a4-49f3970dbcd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd710dd7ca454d819aaf50dc7fc7372f', 'request_id': 'req-891dcb3c-8718-4a16-a2a4-49f3970dbcd0'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:43.477483 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1131/2277] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:43.482937 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf939e55-40cc-4943-981c-2b59dd1463a6 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:43.485466 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-cf939e55-40cc-4943-981c-2b59dd1463a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:43.516879 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-891dcb3c-8718-4a16-a2a4-49f3970dbcd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd710dd7ca454d819aaf50dc7fc7372f', 'request_id': 'req-891dcb3c-8718-4a16-a2a4-49f3970dbcd0'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:43.517728 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf939e55-40cc-4943-981c-2b59dd1463a6 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:43.523257 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-891dcb3c-8718-4a16-a2a4-49f3970dbcd0 admin admin] The token cache is being invalidate because project fd710dd7ca454d819aaf50dc7fc7372f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:51:43.523688 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-891dcb3c-8718-4a16-a2a4-49f3970dbcd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:43.524449 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-891dcb3c-8718-4a16-a2a4-49f3970dbcd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:43.531059 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1147/2278] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:51:43 2022] DELETE /identity/v3/projects/fd710dd7ca454d819aaf50dc7fc7372f => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:43.534336 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f4bab-c0d8-4319-9f76-bb2622e86fa5 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:43.534336 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f4bab-c0d8-4319-9f76-bb2622e86fa5 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:43.534876 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f4bab-c0d8-4319-9f76-bb2622e86fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:43.585886 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-7f1f4bab-c0d8-4319-9f76-bb2622e86fa5 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:43.587523 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-7f1f4bab-c0d8-4319-9f76-bb2622e86fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:43.622760 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d34efa-a6a5-4dc0-9499-c191813873c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:43.624515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d34efa-a6a5-4dc0-9499-c191813873c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:43.625013 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d34efa-a6a5-4dc0-9499-c191813873c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:43.625857 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d34efa-a6a5-4dc0-9499-c191813873c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:43.626839 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d34efa-a6a5-4dc0-9499-c191813873c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:43.630453 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d34efa-a6a5-4dc0-9499-c191813873c9 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:43.670921 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1148/2279] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:43.684550 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238b8a1-bcc5-4c9b-8428-2fbb108cd767 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:43.684861 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238b8a1-bcc5-4c9b-8428-2fbb108cd767 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:43.685200 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238b8a1-bcc5-4c9b-8428-2fbb108cd767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:43.716220 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-7238b8a1-bcc5-4c9b-8428-2fbb108cd767 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:43.716220 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-7238b8a1-bcc5-4c9b-8428-2fbb108cd767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:43.720921 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1149/2280] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:43.725953 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7238b8a1-bcc5-4c9b-8428-2fbb108cd767 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:43.735580 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7238b8a1-bcc5-4c9b-8428-2fbb108cd767 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:43.742090 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab27cc8-6368-4a68-b3c5-5da2fd698531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:43.743123 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab27cc8-6368-4a68-b3c5-5da2fd698531 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:43.743123 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab27cc8-6368-4a68-b3c5-5da2fd698531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:43.743300 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab27cc8-6368-4a68-b3c5-5da2fd698531 admin admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:43.743936 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab27cc8-6368-4a68-b3c5-5da2fd698531 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:51:43.744228 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab27cc8-6368-4a68-b3c5-5da2fd698531 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:43.745808 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab27cc8-6368-4a68-b3c5-5da2fd698531 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:43.758626 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1150/2281] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:51:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:51:43.764244 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab27cc8-6368-4a68-b3c5-5da2fd698531 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:43.771575 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1132/2282] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:51:43 2022] POST /identity/v3/projects => generated 379 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:51:43.775823 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab27cc8-6368-4a68-b3c5-5da2fd698531 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:43.776852 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d34efa-a6a5-4dc0-9499-c191813873c9 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:43.780536 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-05d34efa-a6a5-4dc0-9499-c191813873c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:43.861883 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2454562-8d8d-4424-ad54-bd545a525828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:43.862694 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2454562-8d8d-4424-ad54-bd545a525828 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:43.863027 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2454562-8d8d-4424-ad54-bd545a525828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:43.863229 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2454562-8d8d-4424-ad54-bd545a525828 admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:43.864630 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2454562-8d8d-4424-ad54-bd545a525828 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:43.867055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2454562-8d8d-4424-ad54-bd545a525828 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:43.868068 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d34efa-a6a5-4dc0-9499-c191813873c9 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:43.873457 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8463f-3eb3-46ee-9bdb-4287a80671d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:43.874081 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8463f-3eb3-46ee-9bdb-4287a80671d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:43.874362 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8463f-3eb3-46ee-9bdb-4287a80671d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:43.874636 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8463f-3eb3-46ee-9bdb-4287a80671d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:43.875266 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f8463f-3eb3-46ee-9bdb-4287a80671d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:43.878217 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f8463f-3eb3-46ee-9bdb-4287a80671d0 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:43.961539 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:43.967639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1151/2283] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:51:43 2022] POST /identity/v3/projects => generated 383 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:51:43.971405 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1133/2284] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:51:43 2022] POST /identity/v3/users => generated 296 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:51:43.971553 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2454562-8d8d-4424-ad54-bd545a525828 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:43.975999 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8463f-3eb3-46ee-9bdb-4287a80671d0 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:43.996060 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8463f-3eb3-46ee-9bdb-4287a80671d0 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:44.084863 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa011262-e593-4057-b3eb-376e14682082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:44.085645 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa011262-e593-4057-b3eb-376e14682082 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:44.086037 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa011262-e593-4057-b3eb-376e14682082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:44.086172 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2454562-8d8d-4424-ad54-bd545a525828 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:44.086599 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa011262-e593-4057-b3eb-376e14682082 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:44.087508 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa011262-e593-4057-b3eb-376e14682082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:44.090100 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa011262-e593-4057-b3eb-376e14682082 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:44.091343 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa71a4f-9c96-42c6-9736-e8bcb0d5fa8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:44.091971 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa71a4f-9c96-42c6-9736-e8bcb0d5fa8d admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:44.092248 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa71a4f-9c96-42c6-9736-e8bcb0d5fa8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:44.092518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa71a4f-9c96-42c6-9736-e8bcb0d5fa8d admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:44.093151 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa71a4f-9c96-42c6-9736-e8bcb0d5fa8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:44.095361 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa71a4f-9c96-42c6-9736-e8bcb0d5fa8d admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:44.107361 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1134/2285] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:43 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:51:44.116474 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa011262-e593-4057-b3eb-376e14682082 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:44.133465 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa011262-e593-4057-b3eb-376e14682082 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:44.138946 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ea052-af7c-4834-aa1c-4ee7bd0f66a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:44.139655 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ea052-af7c-4834-aa1c-4ee7bd0f66a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:44.139919 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ea052-af7c-4834-aa1c-4ee7bd0f66a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:44.140197 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ea052-af7c-4834-aa1c-4ee7bd0f66a4 admin admin] PATH_INFO: `/v3/projects/255d6627821c48b38f53f9abe4b6e3ae/users/7a5d1b590faa430e8ebb9c7b0f91ec4d/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:44.171821 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947ea052-af7c-4834-aa1c-4ee7bd0f66a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=255d6627821c48b38f53f9abe4b6e3ae, user_id=7a5d1b590faa430e8ebb9c7b0f91ec4d, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:44.186955 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1152/2286] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:51:43 2022] POST /identity/v3/users => generated 292 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:51:44.195758 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947ea052-af7c-4834-aa1c-4ee7bd0f66a4 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:44.197700 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa71a4f-9c96-42c6-9736-e8bcb0d5fa8d admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:44.223366 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa71a4f-9c96-42c6-9736-e8bcb0d5fa8d admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:44.226410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9f09e5-5d5f-49aa-8a9b-746e07271272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:44.226915 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9f09e5-5d5f-49aa-8a9b-746e07271272 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:44.227421 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9f09e5-5d5f-49aa-8a9b-746e07271272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:44.228045 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9f09e5-5d5f-49aa-8a9b-746e07271272 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:44.228840 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9f09e5-5d5f-49aa-8a9b-746e07271272 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:44.230432 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9f09e5-5d5f-49aa-8a9b-746e07271272 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:44.246668 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1153/2287] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:51:44.256108 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9f09e5-5d5f-49aa-8a9b-746e07271272 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:44.268668 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9f09e5-5d5f-49aa-8a9b-746e07271272 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:44.275895 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a237c-1bdf-4151-8d7f-ec1889e9546a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:44.276524 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a237c-1bdf-4151-8d7f-ec1889e9546a admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:44.276809 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a237c-1bdf-4151-8d7f-ec1889e9546a admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:44.277084 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a237c-1bdf-4151-8d7f-ec1889e9546a admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:44.277790 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a237c-1bdf-4151-8d7f-ec1889e9546a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:44.279566 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a237c-1bdf-4151-8d7f-ec1889e9546a admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:44.295304 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1154/2288] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:51:44.297808 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a237c-1bdf-4151-8d7f-ec1889e9546a admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:44.304152 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-5c3a237c-1bdf-4151-8d7f-ec1889e9546a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:44.305300 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1135/2289] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:51:44 2022] PUT /identity/v3/projects/255d6627821c48b38f53f9abe4b6e3ae/users/7a5d1b590faa430e8ebb9c7b0f91ec4d/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:44.306609 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ea052-af7c-4834-aa1c-4ee7bd0f66a4 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:44.381173 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a237c-1bdf-4151-8d7f-ec1889e9546a admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:44.419826 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ea052-af7c-4834-aa1c-4ee7bd0f66a4 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:44.424739 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea43953-60db-4675-9524-df37204f7116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:44.425408 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea43953-60db-4675-9524-df37204f7116 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:44.425682 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea43953-60db-4675-9524-df37204f7116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:44.426511 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea43953-60db-4675-9524-df37204f7116 admin admin] PATH_INFO: `/v3/projects/1ae71b0b669d457ea2f214844b8ce9e6/users/fabdf6e1b0b3432b84185bea29aa4187/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:44.442729 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421160cc-25fd-49e2-80c0-9136519f2c40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:44.443658 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421160cc-25fd-49e2-80c0-9136519f2c40 service cinder] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:44.444015 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421160cc-25fd-49e2-80c0-9136519f2c40 service cinder] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:44.444363 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421160cc-25fd-49e2-80c0-9136519f2c40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:44.446244 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-421160cc-25fd-49e2-80c0-9136519f2c40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:44.446357 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea43953-60db-4675-9524-df37204f7116 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ae71b0b669d457ea2f214844b8ce9e6, user_id=fabdf6e1b0b3432b84185bea29aa4187, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:44.449186 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea43953-60db-4675-9524-df37204f7116 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:44.512955 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421160cc-25fd-49e2-80c0-9136519f2c40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:44.514784 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421160cc-25fd-49e2-80c0-9136519f2c40 service cinder] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:44.525372 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1155/2290] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:51:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:51:44.526267 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421160cc-25fd-49e2-80c0-9136519f2c40 service cinder] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:44.541485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421160cc-25fd-49e2-80c0-9136519f2c40 service cinder] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:44.553768 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736df54-e471-48cf-acf5-0b571b3278f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:44.555029 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736df54-e471-48cf-acf5-0b571b3278f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:44.555979 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736df54-e471-48cf-acf5-0b571b3278f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:44.560155 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736df54-e471-48cf-acf5-0b571b3278f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:44.561054 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3736df54-e471-48cf-acf5-0b571b3278f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:44.562739 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3736df54-e471-48cf-acf5-0b571b3278f9 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:44.564477 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1136/2291] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:51:44 2022] PUT /identity/v3/projects/1ae71b0b669d457ea2f214844b8ce9e6/users/fabdf6e1b0b3432b84185bea29aa4187/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:44.581128 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1156/2292] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:51:44.582868 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25300d-a8ad-4476-b001-e40835b093b0 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:44.583439 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25300d-a8ad-4476-b001-e40835b093b0 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:44.583658 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736df54-e471-48cf-acf5-0b571b3278f9 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:44.584243 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25300d-a8ad-4476-b001-e40835b093b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:44.599771 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736df54-e471-48cf-acf5-0b571b3278f9 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:44.625444 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-ba25300d-a8ad-4476-b001-e40835b093b0 None None] MFA Rules not processed for user `fabdf6e1b0b3432b84185bea29aa4187`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:44.629881 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ba25300d-a8ad-4476-b001-e40835b093b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:44.650347 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:44.670408 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee5f66-944f-481c-a35d-b0bd93ae3a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:44.671095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee5f66-944f-481c-a35d-b0bd93ae3a87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:44.671406 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee5f66-944f-481c-a35d-b0bd93ae3a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:44.671684 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee5f66-944f-481c-a35d-b0bd93ae3a87 admin admin] PATH_INFO: `/v3/projects/255d6627821c48b38f53f9abe4b6e3ae/users/7a5d1b590faa430e8ebb9c7b0f91ec4d/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:44.673574 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeee5f66-944f-481c-a35d-b0bd93ae3a87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=255d6627821c48b38f53f9abe4b6e3ae, user_id=7a5d1b590faa430e8ebb9c7b0f91ec4d, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:44.676801 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeee5f66-944f-481c-a35d-b0bd93ae3a87 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:44.687045 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1157/2293] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:51:44 2022] PUT /identity/v3/projects/255d6627821c48b38f53f9abe4b6e3ae/users/7a5d1b590faa430e8ebb9c7b0f91ec4d/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:44.688276 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b70ce-b04b-4788-b97a-b1b50e085e64 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:44.688559 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b70ce-b04b-4788-b97a-b1b50e085e64 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:44.688832 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b70ce-b04b-4788-b97a-b1b50e085e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:44.709217 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-013b70ce-b04b-4788-b97a-b1b50e085e64 None None] MFA Rules not processed for user `ad5654cfa65f4cfa9e68832b9cd3d129`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:44.710499 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-013b70ce-b04b-4788-b97a-b1b50e085e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:44.719881 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1137/2294] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:44.767902 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1158/2295] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:44 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:45.460952 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25300d-a8ad-4476-b001-e40835b093b0 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:45.463708 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ba25300d-a8ad-4476-b001-e40835b093b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:45.584536 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25300d-a8ad-4476-b001-e40835b093b0 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:45.678344 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d960ee9-e85b-40c4-b1da-466f6afe5c44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:45.678982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d960ee9-e85b-40c4-b1da-466f6afe5c44 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:45.679671 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d960ee9-e85b-40c4-b1da-466f6afe5c44 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:45.679671 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d960ee9-e85b-40c4-b1da-466f6afe5c44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:45.770446 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d960ee9-e85b-40c4-b1da-466f6afe5c44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:45.772355 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d960ee9-e85b-40c4-b1da-466f6afe5c44 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:45.782370 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1138/2296] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:51:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:51:45.825611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b70ce-b04b-4788-b97a-b1b50e085e64 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:46.023751 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b70ce-b04b-4788-b97a-b1b50e085e64 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:46.055688 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171013c5-a5d7-4ba9-bb95-8e6066f8da78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:46.058777 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171013c5-a5d7-4ba9-bb95-8e6066f8da78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:46.058777 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171013c5-a5d7-4ba9-bb95-8e6066f8da78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:46.068934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171013c5-a5d7-4ba9-bb95-8e6066f8da78 admin admin] PATH_INFO: `/v3/users/fabdf6e1b0b3432b84185bea29aa4187` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:46.072546 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171013c5-a5d7-4ba9-bb95-8e6066f8da78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fabdf6e1b0b3432b84185bea29aa4187)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:46.075676 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171013c5-a5d7-4ba9-bb95-8e6066f8da78 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:46.289132 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-171013c5-a5d7-4ba9-bb95-8e6066f8da78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fabdf6e1b0b3432b84185bea29aa4187'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:46.298215 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-171013c5-a5d7-4ba9-bb95-8e6066f8da78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fabdf6e1b0b3432b84185bea29aa4187'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:46.322117 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-171013c5-a5d7-4ba9-bb95-8e6066f8da78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fabdf6e1b0b3432b84185bea29aa4187'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:46.324026 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-171013c5-a5d7-4ba9-bb95-8e6066f8da78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fabdf6e1b0b3432b84185bea29aa4187'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:46.433917 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-171013c5-a5d7-4ba9-bb95-8e6066f8da78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fabdf6e1b0b3432b84185bea29aa4187'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:46.442200 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1159/2297] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:51:45 2022] DELETE /identity/v3/users/fabdf6e1b0b3432b84185bea29aa4187 => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:46.446803 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e261406-db57-4e28-ae1c-5a622e6d7831 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:46.447093 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e261406-db57-4e28-ae1c-5a622e6d7831 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:46.447514 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e261406-db57-4e28-ae1c-5a622e6d7831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:46.521107 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-3e261406-db57-4e28-ae1c-5a622e6d7831 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:46.523042 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-3e261406-db57-4e28-ae1c-5a622e6d7831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:46.633171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1139/2298] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:46.655795 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171013c5-a5d7-4ba9-bb95-8e6066f8da78 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:46.662797 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-171013c5-a5d7-4ba9-bb95-8e6066f8da78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:46.796213 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e261406-db57-4e28-ae1c-5a622e6d7831 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:46.798376 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-3e261406-db57-4e28-ae1c-5a622e6d7831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:46.815198 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:46.835474 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171013c5-a5d7-4ba9-bb95-8e6066f8da78 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:46.849396 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9678157-748c-484f-8ea2-ab532f240aa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:46.852631 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9678157-748c-484f-8ea2-ab532f240aa5 service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:46.852631 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9678157-748c-484f-8ea2-ab532f240aa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:46.852631 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9678157-748c-484f-8ea2-ab532f240aa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:46.876772 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9678157-748c-484f-8ea2-ab532f240aa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:46.880843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9678157-748c-484f-8ea2-ab532f240aa5 service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:46.895214 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1160/2299] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:51:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:51:46.920822 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e261406-db57-4e28-ae1c-5a622e6d7831 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:46.927591 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc870f-6157-49f6-940f-331e274961c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:46.928732 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc870f-6157-49f6-940f-331e274961c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:46.929181 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc870f-6157-49f6-940f-331e274961c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:46.929457 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc870f-6157-49f6-940f-331e274961c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:46.932931 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b5fc870f-6157-49f6-940f-331e274961c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:47.084790 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fc870f-6157-49f6-940f-331e274961c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:47.091042 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fc870f-6157-49f6-940f-331e274961c5 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:47.100185 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:47.109290 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1140/2300] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:51:46 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:51:47.893111 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9678157-748c-484f-8ea2-ab532f240aa5 service neutron] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:47.901129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-b9678157-748c-484f-8ea2-ab532f240aa5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:47.941072 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9678157-748c-484f-8ea2-ab532f240aa5 service neutron] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:47.949967 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cdf007-3a76-4593-a78d-83baa2ed95b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:47.951832 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cdf007-3a76-4593-a78d-83baa2ed95b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:47.952495 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cdf007-3a76-4593-a78d-83baa2ed95b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:47.953387 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cdf007-3a76-4593-a78d-83baa2ed95b1 admin admin] PATH_INFO: `/v3/projects/1ae71b0b669d457ea2f214844b8ce9e6` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:47.958023 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cdf007-3a76-4593-a78d-83baa2ed95b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ae71b0b669d457ea2f214844b8ce9e6)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:47.959834 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cdf007-3a76-4593-a78d-83baa2ed95b1 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:48.084271 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-e2cdf007-3a76-4593-a78d-83baa2ed95b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ae71b0b669d457ea2f214844b8ce9e6', 'request_id': 'req-e2cdf007-3a76-4593-a78d-83baa2ed95b1'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:48.096718 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-e2cdf007-3a76-4593-a78d-83baa2ed95b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ae71b0b669d457ea2f214844b8ce9e6', 'request_id': 'req-e2cdf007-3a76-4593-a78d-83baa2ed95b1'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:48.163395 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-e2cdf007-3a76-4593-a78d-83baa2ed95b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ae71b0b669d457ea2f214844b8ce9e6', 'request_id': 'req-e2cdf007-3a76-4593-a78d-83baa2ed95b1'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:48.193722 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-e2cdf007-3a76-4593-a78d-83baa2ed95b1 admin admin] The token cache is being invalidate because project 1ae71b0b669d457ea2f214844b8ce9e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:51:48.194268 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-e2cdf007-3a76-4593-a78d-83baa2ed95b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:48.195341 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-e2cdf007-3a76-4593-a78d-83baa2ed95b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:48.199010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1161/2301] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:51:47 2022] DELETE /identity/v3/projects/1ae71b0b669d457ea2f214844b8ce9e6 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:48.213174 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46adea-f676-4373-b543-ceb2baadd4f3 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:48.213174 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46adea-f676-4373-b543-ceb2baadd4f3 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:48.213174 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46adea-f676-4373-b543-ceb2baadd4f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:48.250330 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-4e46adea-f676-4373-b543-ceb2baadd4f3 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:48.252416 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4e46adea-f676-4373-b543-ceb2baadd4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:48.354105 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1141/2302] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:48.362041 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebced65c-bf90-42f8-a45b-5fd781824f4e None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:48.362414 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebced65c-bf90-42f8-a45b-5fd781824f4e None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:48.362700 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebced65c-bf90-42f8-a45b-5fd781824f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:48.386760 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-ebced65c-bf90-42f8-a45b-5fd781824f4e None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:48.388567 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-ebced65c-bf90-42f8-a45b-5fd781824f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:48.397673 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1162/2303] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:48.402479 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46adea-f676-4373-b543-ceb2baadd4f3 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:48.414137 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46adea-f676-4373-b543-ceb2baadd4f3 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:48.422821 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f2fb8-2d6e-4598-8a84-75030fcf54d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:48.423523 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f2fb8-2d6e-4598-8a84-75030fcf54d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:48.423813 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f2fb8-2d6e-4598-8a84-75030fcf54d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:48.424316 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f2fb8-2d6e-4598-8a84-75030fcf54d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:48.424969 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802f2fb8-2d6e-4598-8a84-75030fcf54d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:51:48.425268 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802f2fb8-2d6e-4598-8a84-75030fcf54d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:48.427983 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802f2fb8-2d6e-4598-8a84-75030fcf54d1 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:48.441521 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1142/2304] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:51:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:51:48.447739 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebced65c-bf90-42f8-a45b-5fd781824f4e None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:48.464278 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebced65c-bf90-42f8-a45b-5fd781824f4e None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:48.469867 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f43f377-425e-4861-a37b-054e783fdfab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:48.470812 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f43f377-425e-4861-a37b-054e783fdfab admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:48.471189 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f43f377-425e-4861-a37b-054e783fdfab admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:48.475981 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f43f377-425e-4861-a37b-054e783fdfab admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:48.476807 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f43f377-425e-4861-a37b-054e783fdfab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:48.478879 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f43f377-425e-4861-a37b-054e783fdfab admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:48.483809 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:48.571522 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1163/2305] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:51:48 2022] POST /identity/v3/projects => generated 377 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:51:48.575948 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f2fb8-2d6e-4598-8a84-75030fcf54d1 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:48.589334 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f2fb8-2d6e-4598-8a84-75030fcf54d1 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:48.716918 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b478ec-12f9-48b5-9abf-a89db7908a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:48.717703 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b478ec-12f9-48b5-9abf-a89db7908a0d admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:48.718078 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b478ec-12f9-48b5-9abf-a89db7908a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:48.718572 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b478ec-12f9-48b5-9abf-a89db7908a0d admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:48.719342 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b478ec-12f9-48b5-9abf-a89db7908a0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:48.731474 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b478ec-12f9-48b5-9abf-a89db7908a0d admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:48.787421 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:48.817959 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1143/2306] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:51:48 2022] POST /identity/v3/users => generated 289 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:51:48.822118 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f43f377-425e-4861-a37b-054e783fdfab admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:48.837049 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f43f377-425e-4861-a37b-054e783fdfab admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:48.848918 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a85867-8294-4de1-be58-17d35ce9092f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:48.848918 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a85867-8294-4de1-be58-17d35ce9092f admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:48.848918 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a85867-8294-4de1-be58-17d35ce9092f admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:48.848918 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a85867-8294-4de1-be58-17d35ce9092f admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:48.849254 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a85867-8294-4de1-be58-17d35ce9092f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:48.849254 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a85867-8294-4de1-be58-17d35ce9092f admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:48.877280 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1164/2307] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:48 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:51:48.880433 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b478ec-12f9-48b5-9abf-a89db7908a0d admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:48.905368 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b478ec-12f9-48b5-9abf-a89db7908a0d admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:48.914828 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a79c12-cf3b-4384-89d9-c6a596f6472a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:48.915508 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a79c12-cf3b-4384-89d9-c6a596f6472a admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:48.915996 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a79c12-cf3b-4384-89d9-c6a596f6472a admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:48.916340 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a79c12-cf3b-4384-89d9-c6a596f6472a admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:48.917037 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a79c12-cf3b-4384-89d9-c6a596f6472a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:48.918671 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a79c12-cf3b-4384-89d9-c6a596f6472a admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:48.934855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1144/2308] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:48 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:51:48.936299 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a85867-8294-4de1-be58-17d35ce9092f admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:48.947274 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a85867-8294-4de1-be58-17d35ce9092f admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:48.968513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55037e86-9f6f-4ecd-adbe-55f3f49abd06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:48.969377 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55037e86-9f6f-4ecd-adbe-55f3f49abd06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:48.974633 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55037e86-9f6f-4ecd-adbe-55f3f49abd06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:48.974633 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55037e86-9f6f-4ecd-adbe-55f3f49abd06 admin admin] PATH_INFO: `/v3/projects/4dd9748c9d3348109631f9e9eb403247/users/7e0eeb6194a0429e9f901e7c215e7005/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:49.024831 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55037e86-9f6f-4ecd-adbe-55f3f49abd06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dd9748c9d3348109631f9e9eb403247, user_id=7e0eeb6194a0429e9f901e7c215e7005, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:49.032014 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55037e86-9f6f-4ecd-adbe-55f3f49abd06 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:49.118580 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1165/2309] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:51:48 2022] PUT /identity/v3/projects/4dd9748c9d3348109631f9e9eb403247/users/7e0eeb6194a0429e9f901e7c215e7005/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:49.120581 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4dd6c-0915-44be-848e-6e87cb348e4c None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:49.120875 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4dd6c-0915-44be-848e-6e87cb348e4c None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:49.121151 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4dd6c-0915-44be-848e-6e87cb348e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:49.178068 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-aae4dd6c-0915-44be-848e-6e87cb348e4c None None] MFA Rules not processed for user `7e0eeb6194a0429e9f901e7c215e7005`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:49.179480 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-aae4dd6c-0915-44be-848e-6e87cb348e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:49.372825 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1145/2310] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:49.385349 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55037e86-9f6f-4ecd-adbe-55f3f49abd06 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:49.387205 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-55037e86-9f6f-4ecd-adbe-55f3f49abd06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:49.553518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55037e86-9f6f-4ecd-adbe-55f3f49abd06 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:49.562090 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d2798-6bf9-43df-9830-f11aeebe72b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:49.563391 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d2798-6bf9-43df-9830-f11aeebe72b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:49.564044 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d2798-6bf9-43df-9830-f11aeebe72b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:49.564533 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d2798-6bf9-43df-9830-f11aeebe72b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:49.576934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691d2798-6bf9-43df-9830-f11aeebe72b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:49.578546 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691d2798-6bf9-43df-9830-f11aeebe72b3 service cinder] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:49.601406 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1166/2311] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:51:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:51:49.643685 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:50.563668 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:50.866567 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:51.815160 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:52.417260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:52.629693 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4dd6c-0915-44be-848e-6e87cb348e4c None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:52.631477 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-aae4dd6c-0915-44be-848e-6e87cb348e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:52.768602 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4dd6c-0915-44be-848e-6e87cb348e4c None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:52.773569 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd72df16-9f6f-4de7-90ab-530aa8f73f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:52.774218 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72df16-9f6f-4de7-90ab-530aa8f73f03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:52.774501 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72df16-9f6f-4de7-90ab-530aa8f73f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:52.774775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72df16-9f6f-4de7-90ab-530aa8f73f03 admin admin] PATH_INFO: `/v3/users/2fdfdc6002dd42ca8471dcbc608fdfbd` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:52.776071 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd72df16-9f6f-4de7-90ab-530aa8f73f03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fdfdc6002dd42ca8471dcbc608fdfbd)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:52.777985 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd72df16-9f6f-4de7-90ab-530aa8f73f03 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:53.020888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-fd72df16-9f6f-4de7-90ab-530aa8f73f03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fdfdc6002dd42ca8471dcbc608fdfbd'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:53.048984 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-fd72df16-9f6f-4de7-90ab-530aa8f73f03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fdfdc6002dd42ca8471dcbc608fdfbd'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:53.049739 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-fd72df16-9f6f-4de7-90ab-530aa8f73f03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fdfdc6002dd42ca8471dcbc608fdfbd'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:53.076414 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-fd72df16-9f6f-4de7-90ab-530aa8f73f03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fdfdc6002dd42ca8471dcbc608fdfbd'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:53.159049 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-fd72df16-9f6f-4de7-90ab-530aa8f73f03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fdfdc6002dd42ca8471dcbc608fdfbd'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:53.168331 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1146/2312] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:51:52 2022] DELETE /identity/v3/users/2fdfdc6002dd42ca8471dcbc608fdfbd => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:53.171366 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f74a1f-4518-4608-860c-95ccb8696259 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:53.172178 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f74a1f-4518-4608-860c-95ccb8696259 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:53.172463 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f74a1f-4518-4608-860c-95ccb8696259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:53.188460 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:53.233195 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-e5f74a1f-4518-4608-860c-95ccb8696259 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:53.234621 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-e5f74a1f-4518-4608-860c-95ccb8696259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:53.302136 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:53.350759 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1167/2313] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:53.361939 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd72df16-9f6f-4de7-90ab-530aa8f73f03 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:53.375698 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-fd72df16-9f6f-4de7-90ab-530aa8f73f03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:53.454190 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd72df16-9f6f-4de7-90ab-530aa8f73f03 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:53.459503 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820fe083-ac6b-48e3-a056-811f43b42299 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:53.460247 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820fe083-ac6b-48e3-a056-811f43b42299 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:53.460605 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820fe083-ac6b-48e3-a056-811f43b42299 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:53.461072 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820fe083-ac6b-48e3-a056-811f43b42299 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:53.472315 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820fe083-ac6b-48e3-a056-811f43b42299 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:53.474664 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820fe083-ac6b-48e3-a056-811f43b42299 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:53.487857 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1147/2314] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:51:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:51:53.592485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964ad74-5bb5-43cb-bba2-cdbd6da51d0e None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:53.592881 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964ad74-5bb5-43cb-bba2-cdbd6da51d0e None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:53.593192 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964ad74-5bb5-43cb-bba2-cdbd6da51d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:53.632379 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-f964ad74-5bb5-43cb-bba2-cdbd6da51d0e None None] MFA Rules not processed for user `7a5d1b590faa430e8ebb9c7b0f91ec4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:53.633714 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-f964ad74-5bb5-43cb-bba2-cdbd6da51d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:53.661931 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:53.715689 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1168/2315] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:53 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:53.727184 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820fe083-ac6b-48e3-a056-811f43b42299 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:53.728677 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-820fe083-ac6b-48e3-a056-811f43b42299 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:53.848177 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820fe083-ac6b-48e3-a056-811f43b42299 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:53.855055 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbacf03-b8d3-41e5-a07d-1191b1ca4853 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:53.855920 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbacf03-b8d3-41e5-a07d-1191b1ca4853 service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:53.856373 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbacf03-b8d3-41e5-a07d-1191b1ca4853 service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:53.856800 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbacf03-b8d3-41e5-a07d-1191b1ca4853 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:53.859068 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4dbacf03-b8d3-41e5-a07d-1191b1ca4853 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:53.956093 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbacf03-b8d3-41e5-a07d-1191b1ca4853 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:53.959030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbacf03-b8d3-41e5-a07d-1191b1ca4853 service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:53.994462 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1148/2316] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:51:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:51:54.125261 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f964ad74-5bb5-43cb-bba2-cdbd6da51d0e None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:54.126795 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-f964ad74-5bb5-43cb-bba2-cdbd6da51d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:54.140339 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f964ad74-5bb5-43cb-bba2-cdbd6da51d0e None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:54.145501 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c71cd7-8ec9-4590-b301-f2eb6a0a930a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:54.146146 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c71cd7-8ec9-4590-b301-f2eb6a0a930a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:54.146425 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c71cd7-8ec9-4590-b301-f2eb6a0a930a admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:54.146693 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c71cd7-8ec9-4590-b301-f2eb6a0a930a admin admin] PATH_INFO: `/v3/projects/01d9d29160e54fe79fa64f0b70f0a536` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:54.147855 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c71cd7-8ec9-4590-b301-f2eb6a0a930a admin admin] RBAC: Authorizing `identity:delete_project(project_id=01d9d29160e54fe79fa64f0b70f0a536)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:54.149755 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c71cd7-8ec9-4590-b301-f2eb6a0a930a admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:54.285061 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-09c71cd7-8ec9-4590-b301-f2eb6a0a930a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01d9d29160e54fe79fa64f0b70f0a536', 'request_id': 'req-09c71cd7-8ec9-4590-b301-f2eb6a0a930a'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:54.292596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-09c71cd7-8ec9-4590-b301-f2eb6a0a930a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01d9d29160e54fe79fa64f0b70f0a536', 'request_id': 'req-09c71cd7-8ec9-4590-b301-f2eb6a0a930a'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:54.363584 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-09c71cd7-8ec9-4590-b301-f2eb6a0a930a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01d9d29160e54fe79fa64f0b70f0a536', 'request_id': 'req-09c71cd7-8ec9-4590-b301-f2eb6a0a930a'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:54.377274 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-09c71cd7-8ec9-4590-b301-f2eb6a0a930a admin admin] The token cache is being invalidate because project 01d9d29160e54fe79fa64f0b70f0a536 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:51:54.377639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-09c71cd7-8ec9-4590-b301-f2eb6a0a930a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:54.378508 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-09c71cd7-8ec9-4590-b301-f2eb6a0a930a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:54.381398 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1169/2317] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:51:54 2022] DELETE /identity/v3/projects/01d9d29160e54fe79fa64f0b70f0a536 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:54.385515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbacf03-b8d3-41e5-a07d-1191b1ca4853 service nova] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:54.410603 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbacf03-b8d3-41e5-a07d-1191b1ca4853 service nova] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:54.507008 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e873e540-9848-4890-ad09-31daaa7a0e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:54.507723 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873e540-9848-4890-ad09-31daaa7a0e67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:54.507980 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873e540-9848-4890-ad09-31daaa7a0e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:54.508400 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873e540-9848-4890-ad09-31daaa7a0e67 admin admin] PATH_INFO: `/v3/users/9f6ef09c696948e59075dd110bb5904f` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:54.520579 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e873e540-9848-4890-ad09-31daaa7a0e67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f6ef09c696948e59075dd110bb5904f)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:54.520579 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e873e540-9848-4890-ad09-31daaa7a0e67 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:54.815928 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e873e540-9848-4890-ad09-31daaa7a0e67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f6ef09c696948e59075dd110bb5904f'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:54.837424 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e873e540-9848-4890-ad09-31daaa7a0e67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f6ef09c696948e59075dd110bb5904f'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:54.838196 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e873e540-9848-4890-ad09-31daaa7a0e67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f6ef09c696948e59075dd110bb5904f'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:54.846638 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e873e540-9848-4890-ad09-31daaa7a0e67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f6ef09c696948e59075dd110bb5904f'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:54.922894 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e873e540-9848-4890-ad09-31daaa7a0e67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f6ef09c696948e59075dd110bb5904f'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:54.930948 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1149/2318] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:51:54 2022] DELETE /identity/v3/users/9f6ef09c696948e59075dd110bb5904f => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:55.227291 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:55.483749 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:55.535111 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c71cd7-8ec9-4590-b301-f2eb6a0a930a admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:55.537714 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-09c71cd7-8ec9-4590-b301-f2eb6a0a930a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:55.550627 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c71cd7-8ec9-4590-b301-f2eb6a0a930a admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:55.632111 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff349f6d-68fb-4990-aecf-01ec095c390e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:55.633002 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff349f6d-68fb-4990-aecf-01ec095c390e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:55.633434 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff349f6d-68fb-4990-aecf-01ec095c390e admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:55.633880 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff349f6d-68fb-4990-aecf-01ec095c390e admin admin] PATH_INFO: `/v3/projects/65cd267ba9854f5ab7e729ab115318d1` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:55.635175 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff349f6d-68fb-4990-aecf-01ec095c390e admin admin] RBAC: Authorizing `identity:delete_project(project_id=65cd267ba9854f5ab7e729ab115318d1)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:55.637315 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff349f6d-68fb-4990-aecf-01ec095c390e admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:55.819993 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-ff349f6d-68fb-4990-aecf-01ec095c390e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65cd267ba9854f5ab7e729ab115318d1', 'request_id': 'req-ff349f6d-68fb-4990-aecf-01ec095c390e'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:55.827932 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-ff349f6d-68fb-4990-aecf-01ec095c390e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65cd267ba9854f5ab7e729ab115318d1', 'request_id': 'req-ff349f6d-68fb-4990-aecf-01ec095c390e'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:55.975934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-ff349f6d-68fb-4990-aecf-01ec095c390e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65cd267ba9854f5ab7e729ab115318d1', 'request_id': 'req-ff349f6d-68fb-4990-aecf-01ec095c390e'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:55.994440 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-ff349f6d-68fb-4990-aecf-01ec095c390e admin admin] The token cache is being invalidate because project 65cd267ba9854f5ab7e729ab115318d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:51:55.994834 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-ff349f6d-68fb-4990-aecf-01ec095c390e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:55.995530 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-ff349f6d-68fb-4990-aecf-01ec095c390e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:55.999347 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1170/2319] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:51:55 2022] DELETE /identity/v3/projects/65cd267ba9854f5ab7e729ab115318d1 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:56.003835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e873e540-9848-4890-ad09-31daaa7a0e67 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:56.005884 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e873e540-9848-4890-ad09-31daaa7a0e67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:56.018482 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e873e540-9848-4890-ad09-31daaa7a0e67 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:56.079415 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053f03c-097d-4ef4-a6ac-dc6f16c83300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:56.080058 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053f03c-097d-4ef4-a6ac-dc6f16c83300 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:56.080334 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053f03c-097d-4ef4-a6ac-dc6f16c83300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:56.080603 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053f03c-097d-4ef4-a6ac-dc6f16c83300 admin admin] PATH_INFO: `/v3/users/dba7c84c8bed4568802bbaf15f6c2c24` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:56.081785 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3053f03c-097d-4ef4-a6ac-dc6f16c83300 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dba7c84c8bed4568802bbaf15f6c2c24)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:56.085501 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3053f03c-097d-4ef4-a6ac-dc6f16c83300 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:56.332541 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-3053f03c-097d-4ef4-a6ac-dc6f16c83300 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dba7c84c8bed4568802bbaf15f6c2c24'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:56.346876 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-3053f03c-097d-4ef4-a6ac-dc6f16c83300 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dba7c84c8bed4568802bbaf15f6c2c24'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:56.348246 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-3053f03c-097d-4ef4-a6ac-dc6f16c83300 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dba7c84c8bed4568802bbaf15f6c2c24'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:56.359403 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-3053f03c-097d-4ef4-a6ac-dc6f16c83300 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dba7c84c8bed4568802bbaf15f6c2c24'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:56.473597 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-3053f03c-097d-4ef4-a6ac-dc6f16c83300 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dba7c84c8bed4568802bbaf15f6c2c24'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:56.486356 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1150/2320] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:51:56 2022] DELETE /identity/v3/users/dba7c84c8bed4568802bbaf15f6c2c24 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:57.063100 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff349f6d-68fb-4990-aecf-01ec095c390e admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:57.065788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-ff349f6d-68fb-4990-aecf-01ec095c390e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:57.097575 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff349f6d-68fb-4990-aecf-01ec095c390e admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:57.197603 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b1ec8-0772-46e3-bc97-1dea3c8ef344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:57.198404 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b1ec8-0772-46e3-bc97-1dea3c8ef344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:57.198910 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b1ec8-0772-46e3-bc97-1dea3c8ef344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:57.199182 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b1ec8-0772-46e3-bc97-1dea3c8ef344 admin admin] PATH_INFO: `/v3/projects/d6bc5767b2934d85bc754a037a5c97d6` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:57.200543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b1ec8-0772-46e3-bc97-1dea3c8ef344 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6bc5767b2934d85bc754a037a5c97d6)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:57.202695 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b1ec8-0772-46e3-bc97-1dea3c8ef344 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:57.341198 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:57.360778 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-057b1ec8-0772-46e3-bc97-1dea3c8ef344 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6bc5767b2934d85bc754a037a5c97d6', 'request_id': 'req-057b1ec8-0772-46e3-bc97-1dea3c8ef344'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:57.369267 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-057b1ec8-0772-46e3-bc97-1dea3c8ef344 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6bc5767b2934d85bc754a037a5c97d6', 'request_id': 'req-057b1ec8-0772-46e3-bc97-1dea3c8ef344'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:57.495956 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-057b1ec8-0772-46e3-bc97-1dea3c8ef344 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6bc5767b2934d85bc754a037a5c97d6', 'request_id': 'req-057b1ec8-0772-46e3-bc97-1dea3c8ef344'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:57.512368 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-057b1ec8-0772-46e3-bc97-1dea3c8ef344 admin admin] The token cache is being invalidate because project d6bc5767b2934d85bc754a037a5c97d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:51:57.512665 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-057b1ec8-0772-46e3-bc97-1dea3c8ef344 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:57.514000 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-057b1ec8-0772-46e3-bc97-1dea3c8ef344 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:51:57.515678 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1171/2321] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:51:57 2022] DELETE /identity/v3/projects/d6bc5767b2934d85bc754a037a5c97d6 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:57.521255 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e7471-1286-4c8e-9cac-54888a562711 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:57.521718 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e7471-1286-4c8e-9cac-54888a562711 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:57.522212 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e7471-1286-4c8e-9cac-54888a562711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:57.548037 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-5c4e7471-1286-4c8e-9cac-54888a562711 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:57.549485 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-5c4e7471-1286-4c8e-9cac-54888a562711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:57.663551 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1151/2322] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:57.671060 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6353c92-9db1-4c1a-8c1c-33275fc95b20 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:57.671441 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6353c92-9db1-4c1a-8c1c-33275fc95b20 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:57.671774 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6353c92-9db1-4c1a-8c1c-33275fc95b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:57.703892 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-b6353c92-9db1-4c1a-8c1c-33275fc95b20 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:57.705124 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-b6353c92-9db1-4c1a-8c1c-33275fc95b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:57.718965 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1172/2323] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:57.720170 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e7471-1286-4c8e-9cac-54888a562711 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:57.735853 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e7471-1286-4c8e-9cac-54888a562711 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:57.746131 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bef102-2286-473f-8b1a-c023754b4a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:57.746831 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bef102-2286-473f-8b1a-c023754b4a8d admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:57.747117 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bef102-2286-473f-8b1a-c023754b4a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:57.747580 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bef102-2286-473f-8b1a-c023754b4a8d admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:57.748253 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bef102-2286-473f-8b1a-c023754b4a8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:51:57.748547 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bef102-2286-473f-8b1a-c023754b4a8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:57.750424 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bef102-2286-473f-8b1a-c023754b4a8d admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:57.763684 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1152/2324] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:51:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:51:57.768440 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6353c92-9db1-4c1a-8c1c-33275fc95b20 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:57.785091 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6353c92-9db1-4c1a-8c1c-33275fc95b20 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:57.800156 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:57.806197 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5471ce8-788a-42a5-b778-6d6ef6c33543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:57.807402 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5471ce8-788a-42a5-b778-6d6ef6c33543 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:57.807871 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5471ce8-788a-42a5-b778-6d6ef6c33543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:57.808405 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5471ce8-788a-42a5-b778-6d6ef6c33543 admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:57.809717 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5471ce8-788a-42a5-b778-6d6ef6c33543 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:57.815390 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5471ce8-788a-42a5-b778-6d6ef6c33543 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:57.920680 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1173/2325] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:51:57 2022] POST /identity/v3/projects => generated 363 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:51:57.924776 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bef102-2286-473f-8b1a-c023754b4a8d admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:57.937720 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bef102-2286-473f-8b1a-c023754b4a8d admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:57.997454 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaeecd9-51d1-44a3-85c5-f201bf5b722e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:57.998079 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaeecd9-51d1-44a3-85c5-f201bf5b722e admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:57.998357 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaeecd9-51d1-44a3-85c5-f201bf5b722e admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:57.998675 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaeecd9-51d1-44a3-85c5-f201bf5b722e admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:57.999339 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaeecd9-51d1-44a3-85c5-f201bf5b722e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:58.001256 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaeecd9-51d1-44a3-85c5-f201bf5b722e admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:58.129775 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1153/2326] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:51:57 2022] POST /identity/v3/users => generated 282 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:51:58.131148 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5471ce8-788a-42a5-b778-6d6ef6c33543 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:58.143690 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5471ce8-788a-42a5-b778-6d6ef6c33543 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:58.149093 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4639c-9bbb-47cf-a2fb-68af4dc37343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:58.149695 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4639c-9bbb-47cf-a2fb-68af4dc37343 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:58.150051 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4639c-9bbb-47cf-a2fb-68af4dc37343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:58.150327 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4639c-9bbb-47cf-a2fb-68af4dc37343 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:58.150993 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a4639c-9bbb-47cf-a2fb-68af4dc37343 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:58.152621 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a4639c-9bbb-47cf-a2fb-68af4dc37343 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:58.162590 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1174/2327] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:51:58.166613 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaeecd9-51d1-44a3-85c5-f201bf5b722e admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:58.177337 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaeecd9-51d1-44a3-85c5-f201bf5b722e admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:58.182199 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e9b4e-03fb-499a-9604-e4878682b04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:58.182801 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e9b4e-03fb-499a-9604-e4878682b04f admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:58.183075 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e9b4e-03fb-499a-9604-e4878682b04f admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:58.183373 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e9b4e-03fb-499a-9604-e4878682b04f admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:58.184040 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e9b4e-03fb-499a-9604-e4878682b04f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:58.185628 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e9b4e-03fb-499a-9604-e4878682b04f admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:58.197357 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1154/2328] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:51:58.201467 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4639c-9bbb-47cf-a2fb-68af4dc37343 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:58.212304 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4639c-9bbb-47cf-a2fb-68af4dc37343 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:58.217347 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc3c90b-b44e-4909-a974-1ca7f662350a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:58.218026 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc3c90b-b44e-4909-a974-1ca7f662350a admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:58.218281 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc3c90b-b44e-4909-a974-1ca7f662350a admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:58.218590 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc3c90b-b44e-4909-a974-1ca7f662350a admin admin] PATH_INFO: `/v3/projects/27ed41f5786a4a55922ed992f40c477e/users/b56aafd252e44345baeef1adb3f259ac/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:58.236234 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc3c90b-b44e-4909-a974-1ca7f662350a admin admin] RBAC: Authorizing `identity:create_grant(project_id=27ed41f5786a4a55922ed992f40c477e, user_id=b56aafd252e44345baeef1adb3f259ac, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:58.240191 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc3c90b-b44e-4909-a974-1ca7f662350a admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:58.349900 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1175/2329] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:51:58 2022] PUT /identity/v3/projects/27ed41f5786a4a55922ed992f40c477e/users/b56aafd252e44345baeef1adb3f259ac/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:51:58.354597 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a38ba6-46b9-4791-b9a7-1c4641aab55e None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:58.355058 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a38ba6-46b9-4791-b9a7-1c4641aab55e None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:58.355201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a38ba6-46b9-4791-b9a7-1c4641aab55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:58.386318 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-28a38ba6-46b9-4791-b9a7-1c4641aab55e None None] MFA Rules not processed for user `b56aafd252e44345baeef1adb3f259ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:51:58.387891 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-28a38ba6-46b9-4791-b9a7-1c4641aab55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:58.487731 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1155/2330] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:51:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:51:58.502160 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc3c90b-b44e-4909-a974-1ca7f662350a admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:51:58.504298 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-4fc3c90b-b44e-4909-a974-1ca7f662350a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:51:58.567523 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc3c90b-b44e-4909-a974-1ca7f662350a admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:51:58.571309 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd516c07-b890-47d9-b771-3d1489f8cb5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:51:58.571964 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd516c07-b890-47d9-b771-3d1489f8cb5a service cinder] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:51:58.572241 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd516c07-b890-47d9-b771-3d1489f8cb5a service cinder] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:51:58.572556 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd516c07-b890-47d9-b771-3d1489f8cb5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:51:58.583012 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd516c07-b890-47d9-b771-3d1489f8cb5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:51:58.584521 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd516c07-b890-47d9-b771-3d1489f8cb5a service cinder] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:51:58.595676 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1176/2331] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:51:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:51:59.466221 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:51:59.654995 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:52:02.906982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb75b5-2a75-45c6-98e5-3f99d294f8be None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:02.907518 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb75b5-2a75-45c6-98e5-3f99d294f8be None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:02.907649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb75b5-2a75-45c6-98e5-3f99d294f8be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:02.936470 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-34bb75b5-2a75-45c6-98e5-3f99d294f8be None None] MFA Rules not processed for user `2e6e4387a7c24bbc82c22eefe2a3d138`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:52:02.940742 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-34bb75b5-2a75-45c6-98e5-3f99d294f8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:03.010922 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1156/2332] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Oct 3 12:52:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:52:03.077438 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd516c07-b890-47d9-b771-3d1489f8cb5a service cinder] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:03.079180 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-cd516c07-b890-47d9-b771-3d1489f8cb5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:03.157124 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd516c07-b890-47d9-b771-3d1489f8cb5a service cinder] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:03.163602 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd11fd1-fdeb-4c19-b429-c503c6f89e1c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b111d72a34144073917de4f566a58233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:03.164428 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd11fd1-fdeb-4c19-b429-c503c6f89e1c service swift] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:03.165447 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd11fd1-fdeb-4c19-b429-c503c6f89e1c service swift] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:03.165795 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd11fd1-fdeb-4c19-b429-c503c6f89e1c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:03.175031 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd11fd1-fdeb-4c19-b429-c503c6f89e1c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:03.177822 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd11fd1-fdeb-4c19-b429-c503c6f89e1c service swift] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:03.188325 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1177/2333] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Oct 3 12:52:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 12:52:13.003158 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=74,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:52:13.184963 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=66,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:52:20.816311 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb75b5-2a75-45c6-98e5-3f99d294f8be None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:20.822737 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-34bb75b5-2a75-45c6-98e5-3f99d294f8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:20.898783 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb75b5-2a75-45c6-98e5-3f99d294f8be None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:20.904400 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c5c4a-53c6-44f1-b52b-7db5d2ad4ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:20.905279 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c5c4a-53c6-44f1-b52b-7db5d2ad4ebd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:20.905629 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c5c4a-53c6-44f1-b52b-7db5d2ad4ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:20.906004 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c5c4a-53c6-44f1-b52b-7db5d2ad4ebd admin admin] PATH_INFO: `/v3/users/7e0eeb6194a0429e9f901e7c215e7005` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:20.907696 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c5c4a-53c6-44f1-b52b-7db5d2ad4ebd admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e0eeb6194a0429e9f901e7c215e7005)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:20.909727 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c5c4a-53c6-44f1-b52b-7db5d2ad4ebd admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:21.135566 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-280c5c4a-53c6-44f1-b52b-7db5d2ad4ebd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e0eeb6194a0429e9f901e7c215e7005'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:21.154838 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-280c5c4a-53c6-44f1-b52b-7db5d2ad4ebd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e0eeb6194a0429e9f901e7c215e7005'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:21.161347 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-280c5c4a-53c6-44f1-b52b-7db5d2ad4ebd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e0eeb6194a0429e9f901e7c215e7005'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:21.168995 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-280c5c4a-53c6-44f1-b52b-7db5d2ad4ebd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e0eeb6194a0429e9f901e7c215e7005'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:21.255351 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-280c5c4a-53c6-44f1-b52b-7db5d2ad4ebd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e0eeb6194a0429e9f901e7c215e7005'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:21.262247 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1157/2334] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:52:20 2022] DELETE /identity/v3/users/7e0eeb6194a0429e9f901e7c215e7005 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:52:21.267164 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec58be8-586a-4aae-8a9b-826ebbc84f38 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:21.267733 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec58be8-586a-4aae-8a9b-826ebbc84f38 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:21.268045 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec58be8-586a-4aae-8a9b-826ebbc84f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:21.292054 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-9ec58be8-586a-4aae-8a9b-826ebbc84f38 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:52:21.293581 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-9ec58be8-586a-4aae-8a9b-826ebbc84f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:21.373768 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1178/2335] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:52:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:52:21.386369 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c5c4a-53c6-44f1-b52b-7db5d2ad4ebd admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:21.388059 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-280c5c4a-53c6-44f1-b52b-7db5d2ad4ebd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:21.460914 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c5c4a-53c6-44f1-b52b-7db5d2ad4ebd admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:21.465277 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ec31a-1331-4d68-9508-792fed4a72ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:21.466139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ec31a-1331-4d68-9508-792fed4a72ae service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:21.466389 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ec31a-1331-4d68-9508-792fed4a72ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:21.466674 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ec31a-1331-4d68-9508-792fed4a72ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:21.479661 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9ec31a-1331-4d68-9508-792fed4a72ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:21.481184 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9ec31a-1331-4d68-9508-792fed4a72ae service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:21.492286 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1158/2336] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:52:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:52:22.065174 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec58be8-586a-4aae-8a9b-826ebbc84f38 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:22.069197 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-9ec58be8-586a-4aae-8a9b-826ebbc84f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:22.083831 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec58be8-586a-4aae-8a9b-826ebbc84f38 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:22.089863 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ffce6-616d-405a-b6c2-659376801ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:22.090865 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ffce6-616d-405a-b6c2-659376801ced admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:22.091153 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ffce6-616d-405a-b6c2-659376801ced admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:22.091561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ffce6-616d-405a-b6c2-659376801ced admin admin] PATH_INFO: `/v3/projects/4dd9748c9d3348109631f9e9eb403247` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:22.098889 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170ffce6-616d-405a-b6c2-659376801ced admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dd9748c9d3348109631f9e9eb403247)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:22.099640 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170ffce6-616d-405a-b6c2-659376801ced admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:22.257958 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-170ffce6-616d-405a-b6c2-659376801ced admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dd9748c9d3348109631f9e9eb403247', 'request_id': 'req-170ffce6-616d-405a-b6c2-659376801ced'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:22.265244 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-170ffce6-616d-405a-b6c2-659376801ced admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dd9748c9d3348109631f9e9eb403247', 'request_id': 'req-170ffce6-616d-405a-b6c2-659376801ced'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:22.323677 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-170ffce6-616d-405a-b6c2-659376801ced admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dd9748c9d3348109631f9e9eb403247', 'request_id': 'req-170ffce6-616d-405a-b6c2-659376801ced'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:22.340681 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-170ffce6-616d-405a-b6c2-659376801ced admin admin] The token cache is being invalidate because project 4dd9748c9d3348109631f9e9eb403247 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:52:22.341120 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-170ffce6-616d-405a-b6c2-659376801ced admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:22.342039 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-170ffce6-616d-405a-b6c2-659376801ced admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:22.344012 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1179/2337] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:52:22 2022] DELETE /identity/v3/projects/4dd9748c9d3348109631f9e9eb403247 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:52:22.349599 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1eba62-3ddf-4fc5-991c-ba102648bc10 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:22.350746 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1eba62-3ddf-4fc5-991c-ba102648bc10 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:22.351026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1eba62-3ddf-4fc5-991c-ba102648bc10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:22.374765 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-4a1eba62-3ddf-4fc5-991c-ba102648bc10 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:52:22.376027 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4a1eba62-3ddf-4fc5-991c-ba102648bc10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:22.399461 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:52:22.467426 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1159/2338] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:52:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:52:22.478986 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849132a-a560-4d52-8524-a5dabe8db4e0 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:22.479565 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849132a-a560-4d52-8524-a5dabe8db4e0 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:22.479840 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849132a-a560-4d52-8524-a5dabe8db4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:22.505055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-f849132a-a560-4d52-8524-a5dabe8db4e0 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:52:22.506860 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-f849132a-a560-4d52-8524-a5dabe8db4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:22.521512 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1eba62-3ddf-4fc5-991c-ba102648bc10 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:22.523474 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1180/2339] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:52:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:52:22.533291 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1eba62-3ddf-4fc5-991c-ba102648bc10 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:22.539594 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9584c56-185a-4b4f-b724-2f3d64e9ec0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:22.540571 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9584c56-185a-4b4f-b724-2f3d64e9ec0d admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:22.540941 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9584c56-185a-4b4f-b724-2f3d64e9ec0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:22.541265 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9584c56-185a-4b4f-b724-2f3d64e9ec0d admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:22.541879 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9584c56-185a-4b4f-b724-2f3d64e9ec0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:52:22.542182 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9584c56-185a-4b4f-b724-2f3d64e9ec0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:22.543863 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9584c56-185a-4b4f-b724-2f3d64e9ec0d admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:22.557727 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1160/2340] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:52:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:52:22.562397 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f849132a-a560-4d52-8524-a5dabe8db4e0 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:22.574736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f849132a-a560-4d52-8524-a5dabe8db4e0 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:22.580116 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6eff57-e5cd-401d-b50d-042ac0867885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:22.580723 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6eff57-e5cd-401d-b50d-042ac0867885 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:22.581055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6eff57-e5cd-401d-b50d-042ac0867885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:22.581424 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6eff57-e5cd-401d-b50d-042ac0867885 admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:22.582179 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6eff57-e5cd-401d-b50d-042ac0867885 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:22.584854 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6eff57-e5cd-401d-b50d-042ac0867885 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:22.664299 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1181/2341] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:52:22 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:52:22.668435 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9584c56-185a-4b4f-b724-2f3d64e9ec0d admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:22.680594 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9584c56-185a-4b4f-b724-2f3d64e9ec0d admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:22.752307 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad5955f-1b47-43fc-acfc-24c8ed00fa0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:22.753060 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5955f-1b47-43fc-acfc-24c8ed00fa0a admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:22.753342 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5955f-1b47-43fc-acfc-24c8ed00fa0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:22.753645 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5955f-1b47-43fc-acfc-24c8ed00fa0a admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:22.754346 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad5955f-1b47-43fc-acfc-24c8ed00fa0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:22.756406 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad5955f-1b47-43fc-acfc-24c8ed00fa0a admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:22.877362 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1161/2342] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:52:22 2022] POST /identity/v3/users => generated 289 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:52:22.882289 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6eff57-e5cd-401d-b50d-042ac0867885 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:22.898252 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6eff57-e5cd-401d-b50d-042ac0867885 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:22.917497 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e0e8bf-3e49-4183-b151-f835d9452683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:22.918421 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0e8bf-3e49-4183-b151-f835d9452683 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:22.918704 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0e8bf-3e49-4183-b151-f835d9452683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:22.919130 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0e8bf-3e49-4183-b151-f835d9452683 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:22.919852 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e0e8bf-3e49-4183-b151-f835d9452683 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:22.921540 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e0e8bf-3e49-4183-b151-f835d9452683 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:22.943426 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1182/2343] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:52:22.948083 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad5955f-1b47-43fc-acfc-24c8ed00fa0a admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:22.963729 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad5955f-1b47-43fc-acfc-24c8ed00fa0a admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:22.971036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131d3cf-0def-4a1f-b998-cdd0018e3508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:22.972304 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131d3cf-0def-4a1f-b998-cdd0018e3508 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:22.972304 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131d3cf-0def-4a1f-b998-cdd0018e3508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:22.972905 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131d3cf-0def-4a1f-b998-cdd0018e3508 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:22.974189 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131d3cf-0def-4a1f-b998-cdd0018e3508 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:22.976003 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131d3cf-0def-4a1f-b998-cdd0018e3508 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:22.995990 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1162/2344] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:52:23.000758 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e0e8bf-3e49-4183-b151-f835d9452683 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:23.013685 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e0e8bf-3e49-4183-b151-f835d9452683 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:23.020411 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04028c43-8925-410a-8f4f-d21fc71f68d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:23.021392 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04028c43-8925-410a-8f4f-d21fc71f68d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:23.021672 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04028c43-8925-410a-8f4f-d21fc71f68d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:23.022048 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04028c43-8925-410a-8f4f-d21fc71f68d5 admin admin] PATH_INFO: `/v3/projects/5002df604f7c489f98db223cbf8f997d/users/5febdb62410749c2b7611f8166dc3370/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:23.038784 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04028c43-8925-410a-8f4f-d21fc71f68d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5002df604f7c489f98db223cbf8f997d, user_id=5febdb62410749c2b7611f8166dc3370, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:23.041854 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04028c43-8925-410a-8f4f-d21fc71f68d5 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:23.121096 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1183/2345] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:52:22 2022] PUT /identity/v3/projects/5002df604f7c489f98db223cbf8f997d/users/5febdb62410749c2b7611f8166dc3370/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:52:23.139072 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4cd09-18b4-4f6d-869e-24e4fc673f12 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:23.140268 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4cd09-18b4-4f6d-869e-24e4fc673f12 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:23.140268 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4cd09-18b4-4f6d-869e-24e4fc673f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:23.174243 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-c8b4cd09-18b4-4f6d-869e-24e4fc673f12 None None] MFA Rules not processed for user `5febdb62410749c2b7611f8166dc3370`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:52:23.175774 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-c8b4cd09-18b4-4f6d-869e-24e4fc673f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:23.279252 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1163/2346] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:52:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:52:23.283675 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04028c43-8925-410a-8f4f-d21fc71f68d5 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:23.306238 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:52:23.351435 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04028c43-8925-410a-8f4f-d21fc71f68d5 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:23.357499 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f2a6e-97e0-473e-b7e4-3a273bb9454a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:23.358217 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f2a6e-97e0-473e-b7e4-3a273bb9454a admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:23.358496 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f2a6e-97e0-473e-b7e4-3a273bb9454a admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:23.358872 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f2a6e-97e0-473e-b7e4-3a273bb9454a admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:23.359679 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00f2a6e-97e0-473e-b7e4-3a273bb9454a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:23.362543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00f2a6e-97e0-473e-b7e4-3a273bb9454a admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:23.463734 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1184/2347] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:52:23 2022] POST /identity/v3/projects => generated 377 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:52:23.467879 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b4cd09-18b4-4f6d-869e-24e4fc673f12 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:23.480311 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b4cd09-18b4-4f6d-869e-24e4fc673f12 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:23.494984 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:52:23.545404 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba772db-0812-4358-a4ac-f58e97a12e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:23.546397 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba772db-0812-4358-a4ac-f58e97a12e69 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:23.546822 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba772db-0812-4358-a4ac-f58e97a12e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:23.547159 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba772db-0812-4358-a4ac-f58e97a12e69 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:23.547948 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba772db-0812-4358-a4ac-f58e97a12e69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:23.550521 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba772db-0812-4358-a4ac-f58e97a12e69 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:23.688254 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1164/2348] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:52:23 2022] POST /identity/v3/users => generated 289 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:52:23.700261 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f2a6e-97e0-473e-b7e4-3a273bb9454a admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:23.716478 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f2a6e-97e0-473e-b7e4-3a273bb9454a admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:23.737384 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fac5ba-dbfa-44cf-a68f-7e204fe313c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:23.738237 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fac5ba-dbfa-44cf-a68f-7e204fe313c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:23.745417 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fac5ba-dbfa-44cf-a68f-7e204fe313c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:23.745417 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fac5ba-dbfa-44cf-a68f-7e204fe313c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:23.745417 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fac5ba-dbfa-44cf-a68f-7e204fe313c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:23.745417 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fac5ba-dbfa-44cf-a68f-7e204fe313c1 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:23.758584 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1185/2349] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 12:52:23.762921 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba772db-0812-4358-a4ac-f58e97a12e69 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:23.777450 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba772db-0812-4358-a4ac-f58e97a12e69 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:23.785758 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc5133-ba28-4026-a386-7df58b670da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:23.786590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc5133-ba28-4026-a386-7df58b670da6 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:23.786994 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc5133-ba28-4026-a386-7df58b670da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:23.787346 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc5133-ba28-4026-a386-7df58b670da6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:23.788110 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fc5133-ba28-4026-a386-7df58b670da6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:23.789716 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fc5133-ba28-4026-a386-7df58b670da6 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:23.805067 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1165/2350] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:52:23.806370 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fac5ba-dbfa-44cf-a68f-7e204fe313c1 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:23.839369 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fac5ba-dbfa-44cf-a68f-7e204fe313c1 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:23.857365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d60d497-56f7-49c2-8c07-5d1bf5f2ff42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:23.858060 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60d497-56f7-49c2-8c07-5d1bf5f2ff42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:23.858342 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60d497-56f7-49c2-8c07-5d1bf5f2ff42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:23.858617 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60d497-56f7-49c2-8c07-5d1bf5f2ff42 admin admin] PATH_INFO: `/v3/projects/0dcfe89166a74fe58c5bbebc676d58f9/users/58bda88187a043038ef8b4be93a7d95d/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:23.876009 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d60d497-56f7-49c2-8c07-5d1bf5f2ff42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dcfe89166a74fe58c5bbebc676d58f9, user_id=58bda88187a043038ef8b4be93a7d95d, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:23.878707 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d60d497-56f7-49c2-8c07-5d1bf5f2ff42 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:23.940946 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1186/2351] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:52:23 2022] PUT /identity/v3/projects/0dcfe89166a74fe58c5bbebc676d58f9/users/58bda88187a043038ef8b4be93a7d95d/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:52:23.951181 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18073831-7a09-4da7-866f-9343f7c99e21 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:23.951519 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18073831-7a09-4da7-866f-9343f7c99e21 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:23.951828 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18073831-7a09-4da7-866f-9343f7c99e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:23.981845 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-18073831-7a09-4da7-866f-9343f7c99e21 None None] MFA Rules not processed for user `58bda88187a043038ef8b4be93a7d95d`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:52:23.986975 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-18073831-7a09-4da7-866f-9343f7c99e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:24.075478 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1166/2352] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:52:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:52:24.077724 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d60d497-56f7-49c2-8c07-5d1bf5f2ff42 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:24.214144 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d60d497-56f7-49c2-8c07-5d1bf5f2ff42 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:24.221943 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be28be-3d34-4356-a66a-ca537ce2c79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:24.222563 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be28be-3d34-4356-a66a-ca537ce2c79c admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:24.223515 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be28be-3d34-4356-a66a-ca537ce2c79c admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:24.224015 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be28be-3d34-4356-a66a-ca537ce2c79c admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:24.224779 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0be28be-3d34-4356-a66a-ca537ce2c79c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:24.226688 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0be28be-3d34-4356-a66a-ca537ce2c79c admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:24.340467 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1187/2353] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:52:24 2022] POST /identity/v3/projects => generated 377 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:52:24.345014 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18073831-7a09-4da7-866f-9343f7c99e21 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:24.360407 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18073831-7a09-4da7-866f-9343f7c99e21 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:24.435653 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4a582-7529-4167-80f4-66bdf8421559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:24.436600 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4a582-7529-4167-80f4-66bdf8421559 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:24.436896 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4a582-7529-4167-80f4-66bdf8421559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:24.437262 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4a582-7529-4167-80f4-66bdf8421559 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:24.437903 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca4a582-7529-4167-80f4-66bdf8421559 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:24.439902 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca4a582-7529-4167-80f4-66bdf8421559 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:24.554915 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1167/2354] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:52:24 2022] POST /identity/v3/users => generated 295 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:52:24.558754 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be28be-3d34-4356-a66a-ca537ce2c79c admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:24.574479 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be28be-3d34-4356-a66a-ca537ce2c79c admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:24.580041 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d20b370-da1d-40f9-8f47-48a7ad31d308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:24.581104 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20b370-da1d-40f9-8f47-48a7ad31d308 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:24.581679 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20b370-da1d-40f9-8f47-48a7ad31d308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:24.582902 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20b370-da1d-40f9-8f47-48a7ad31d308 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:24.584661 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d20b370-da1d-40f9-8f47-48a7ad31d308 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:24.586611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d20b370-da1d-40f9-8f47-48a7ad31d308 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:24.603916 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1188/2355] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:52:24.608086 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4a582-7529-4167-80f4-66bdf8421559 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:24.624760 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4a582-7529-4167-80f4-66bdf8421559 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:24.630709 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53939daf-30b1-4be9-a2f3-343dcb66c64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:24.631508 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53939daf-30b1-4be9-a2f3-343dcb66c64d admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:24.632435 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53939daf-30b1-4be9-a2f3-343dcb66c64d admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:24.632435 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53939daf-30b1-4be9-a2f3-343dcb66c64d admin admin] PATH_INFO: `/v3/projects/aa9769292c00415399b0358aa260f920/users/a53ea1bae1914a69a9e1f94c9ae29a6f/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:24.653704 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:52:24.658396 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53939daf-30b1-4be9-a2f3-343dcb66c64d admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa9769292c00415399b0358aa260f920, user_id=a53ea1bae1914a69a9e1f94c9ae29a6f, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:24.662485 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53939daf-30b1-4be9-a2f3-343dcb66c64d admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:24.734776 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1168/2356] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:52:24 2022] PUT /identity/v3/projects/aa9769292c00415399b0358aa260f920/users/a53ea1bae1914a69a9e1f94c9ae29a6f/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:52:24.736114 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d20b370-da1d-40f9-8f47-48a7ad31d308 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:24.751231 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d20b370-da1d-40f9-8f47-48a7ad31d308 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:24.803867 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:52:24.821530 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252b8765-bb39-4c0d-b1c0-6a2e300caabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:24.823275 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b8765-bb39-4c0d-b1c0-6a2e300caabd admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:24.823275 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b8765-bb39-4c0d-b1c0-6a2e300caabd admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:24.823275 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b8765-bb39-4c0d-b1c0-6a2e300caabd admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:24.824166 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252b8765-bb39-4c0d-b1c0-6a2e300caabd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:24.826335 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252b8765-bb39-4c0d-b1c0-6a2e300caabd admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:24.838998 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1189/2357] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:52:24.843754 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53939daf-30b1-4be9-a2f3-343dcb66c64d admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:24.858165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53939daf-30b1-4be9-a2f3-343dcb66c64d admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:24.863886 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f45e08-53d1-411b-8834-a503869884e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:24.865259 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f45e08-53d1-411b-8834-a503869884e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:24.865932 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f45e08-53d1-411b-8834-a503869884e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:24.866474 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f45e08-53d1-411b-8834-a503869884e9 admin admin] PATH_INFO: `/v3/projects/aa9769292c00415399b0358aa260f920/users/a53ea1bae1914a69a9e1f94c9ae29a6f/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:24.870140 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f45e08-53d1-411b-8834-a503869884e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa9769292c00415399b0358aa260f920, user_id=a53ea1bae1914a69a9e1f94c9ae29a6f, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:24.875044 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f45e08-53d1-411b-8834-a503869884e9 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:24.899356 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1169/2358] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:52:24 2022] PUT /identity/v3/projects/aa9769292c00415399b0358aa260f920/users/a53ea1bae1914a69a9e1f94c9ae29a6f/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:52:24.909972 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39604e-1805-4a3e-9353-cb210e391db9 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:24.914115 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39604e-1805-4a3e-9353-cb210e391db9 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:24.914115 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39604e-1805-4a3e-9353-cb210e391db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:24.955032 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-fa39604e-1805-4a3e-9353-cb210e391db9 None None] MFA Rules not processed for user `a53ea1bae1914a69a9e1f94c9ae29a6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:52:24.956399 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-fa39604e-1805-4a3e-9353-cb210e391db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:25.056795 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1190/2359] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:52:24 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:52:25.069281 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f45e08-53d1-411b-8834-a503869884e9 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:25.071451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e7f45e08-53d1-411b-8834-a503869884e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:25.146397 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f45e08-53d1-411b-8834-a503869884e9 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:25.151248 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c1e5f4-b8ce-4d4a-888e-73bc40a55e66 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:25.151710 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c1e5f4-b8ce-4d4a-888e-73bc40a55e66 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:25.152007 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c1e5f4-b8ce-4d4a-888e-73bc40a55e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:25.152448 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd36b18-ae07-46d9-ad3b-098d8e33e8b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:25.153300 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd36b18-ae07-46d9-ad3b-098d8e33e8b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:25.153646 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd36b18-ae07-46d9-ad3b-098d8e33e8b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:25.154130 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd36b18-ae07-46d9-ad3b-098d8e33e8b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:25.180047 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-40c1e5f4-b8ce-4d4a-888e-73bc40a55e66 None None] MFA Rules not processed for user `2e6e4387a7c24bbc82c22eefe2a3d138`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:52:25.180047 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-40c1e5f4-b8ce-4d4a-888e-73bc40a55e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:25.279537 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd36b18-ae07-46d9-ad3b-098d8e33e8b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:25.281058 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd36b18-ae07-46d9-ad3b-098d8e33e8b2 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:25.295666 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1170/2360] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:52:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:52:25.329472 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1191/2361] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Oct 3 12:52:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:52:25.339579 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd36b18-ae07-46d9-ad3b-098d8e33e8b2 service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:25.341430 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-efd36b18-ae07-46d9-ad3b-098d8e33e8b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:25.474544 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd36b18-ae07-46d9-ad3b-098d8e33e8b2 service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:25.487971 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e652e1b-be09-4e8b-94d7-ac24d22064d8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b111d72a34144073917de4f566a58233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:25.490255 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e652e1b-be09-4e8b-94d7-ac24d22064d8 service swift] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:25.490773 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e652e1b-be09-4e8b-94d7-ac24d22064d8 service swift] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:25.491485 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e652e1b-be09-4e8b-94d7-ac24d22064d8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:25.507522 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e652e1b-be09-4e8b-94d7-ac24d22064d8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:25.509245 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e652e1b-be09-4e8b-94d7-ac24d22064d8 service swift] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:25.535141 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1171/2362] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Oct 3 12:52:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 12:52:27.720192 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c1e5f4-b8ce-4d4a-888e-73bc40a55e66 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:27.753928 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c1e5f4-b8ce-4d4a-888e-73bc40a55e66 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:27.753928 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee538ba-b2a6-4fc4-a7f0-206db650e6a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:27.753928 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee538ba-b2a6-4fc4-a7f0-206db650e6a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:27.753928 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee538ba-b2a6-4fc4-a7f0-206db650e6a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:27.754929 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee538ba-b2a6-4fc4-a7f0-206db650e6a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:27.813044 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee538ba-b2a6-4fc4-a7f0-206db650e6a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:27.814495 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee538ba-b2a6-4fc4-a7f0-206db650e6a1 service cinder] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:27.827681 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1192/2363] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:52:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:52:29.996802 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e652e1b-be09-4e8b-94d7-ac24d22064d8 service swift] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:30.015562 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e652e1b-be09-4e8b-94d7-ac24d22064d8 service swift] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:30.030478 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a062870-b110-4cad-8bf3-cdd3684cce44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:30.033288 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a062870-b110-4cad-8bf3-cdd3684cce44 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:30.034329 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a062870-b110-4cad-8bf3-cdd3684cce44 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:30.034899 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a062870-b110-4cad-8bf3-cdd3684cce44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:30.063481 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a062870-b110-4cad-8bf3-cdd3684cce44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:30.066327 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a062870-b110-4cad-8bf3-cdd3684cce44 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:30.099979 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1172/2364] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:52:29 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:52:34.621677 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d67ac3-bdea-4007-b8f1-b5d345c572fc None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:34.622635 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d67ac3-bdea-4007-b8f1-b5d345c572fc None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:34.623172 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d67ac3-bdea-4007-b8f1-b5d345c572fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:34.684754 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-19d67ac3-bdea-4007-b8f1-b5d345c572fc None None] MFA Rules not processed for user `2e6e4387a7c24bbc82c22eefe2a3d138`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:52:34.686000 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-19d67ac3-bdea-4007-b8f1-b5d345c572fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:34.708154 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1193/2365] 10.222.0.33 () {58 vars in 967 bytes} [Mon Oct 3 12:52:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:52:34.724953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a062870-b110-4cad-8bf3-cdd3684cce44 service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:34.743398 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a062870-b110-4cad-8bf3-cdd3684cce44 service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:34.759197 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cb8b89-83ab-4d2b-a69d-a45ab0eaf73c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b111d72a34144073917de4f566a58233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:34.760145 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb8b89-83ab-4d2b-a69d-a45ab0eaf73c service swift] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:34.760425 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb8b89-83ab-4d2b-a69d-a45ab0eaf73c service swift] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:34.760698 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb8b89-83ab-4d2b-a69d-a45ab0eaf73c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:34.769013 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:52:34.773837 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cb8b89-83ab-4d2b-a69d-a45ab0eaf73c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:34.775465 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cb8b89-83ab-4d2b-a69d-a45ab0eaf73c service swift] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:34.785864 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1173/2366] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Oct 3 12:52:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 12:52:35.449715 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d67ac3-bdea-4007-b8f1-b5d345c572fc None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:35.454300 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:52:35.545609 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d67ac3-bdea-4007-b8f1-b5d345c572fc None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:35.559731 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f84175-197c-4721-988e-ad13293181ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:35.560404 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f84175-197c-4721-988e-ad13293181ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:35.560681 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f84175-197c-4721-988e-ad13293181ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:35.560954 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f84175-197c-4721-988e-ad13293181ef admin admin] PATH_INFO: `/v3/users/5febdb62410749c2b7611f8166dc3370` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:35.564362 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f84175-197c-4721-988e-ad13293181ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=5febdb62410749c2b7611f8166dc3370)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:35.567128 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f84175-197c-4721-988e-ad13293181ef admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:35.791172 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-28f84175-197c-4721-988e-ad13293181ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5febdb62410749c2b7611f8166dc3370'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:35.810032 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-28f84175-197c-4721-988e-ad13293181ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5febdb62410749c2b7611f8166dc3370'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:35.829188 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-28f84175-197c-4721-988e-ad13293181ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5febdb62410749c2b7611f8166dc3370'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:35.830460 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-28f84175-197c-4721-988e-ad13293181ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5febdb62410749c2b7611f8166dc3370'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:35.918182 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-28f84175-197c-4721-988e-ad13293181ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5febdb62410749c2b7611f8166dc3370'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:35.927164 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1194/2367] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:52:35 2022] DELETE /identity/v3/users/5febdb62410749c2b7611f8166dc3370 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:52:35.931509 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857e797-a13a-4f5b-866e-b99c98df3105 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:35.932537 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857e797-a13a-4f5b-866e-b99c98df3105 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:35.932987 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857e797-a13a-4f5b-866e-b99c98df3105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:35.954751 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-c857e797-a13a-4f5b-866e-b99c98df3105 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:52:35.956017 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-c857e797-a13a-4f5b-866e-b99c98df3105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:36.033197 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1174/2368] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:52:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:52:36.041599 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f84175-197c-4721-988e-ad13293181ef admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:36.043340 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-28f84175-197c-4721-988e-ad13293181ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:36.151495 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f84175-197c-4721-988e-ad13293181ef admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:36.155621 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2a3d8-b51f-43a4-9d92-8ee07207137c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:36.156244 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2a3d8-b51f-43a4-9d92-8ee07207137c service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:36.156533 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2a3d8-b51f-43a4-9d92-8ee07207137c service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:36.156799 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2a3d8-b51f-43a4-9d92-8ee07207137c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:36.168141 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c2a3d8-b51f-43a4-9d92-8ee07207137c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:36.169865 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c2a3d8-b51f-43a4-9d92-8ee07207137c service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:36.180945 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1195/2369] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:52:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:52:37.052012 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c857e797-a13a-4f5b-866e-b99c98df3105 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:37.061657 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-c857e797-a13a-4f5b-866e-b99c98df3105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:37.074770 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c857e797-a13a-4f5b-866e-b99c98df3105 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:37.079443 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de2218-fe94-45d0-af42-21de225b61b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:37.080082 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de2218-fe94-45d0-af42-21de225b61b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:37.080361 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de2218-fe94-45d0-af42-21de225b61b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:37.080635 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de2218-fe94-45d0-af42-21de225b61b5 admin admin] PATH_INFO: `/v3/projects/5002df604f7c489f98db223cbf8f997d` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:37.081983 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de2218-fe94-45d0-af42-21de225b61b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5002df604f7c489f98db223cbf8f997d)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:37.084300 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de2218-fe94-45d0-af42-21de225b61b5 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:37.211411 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-01de2218-fe94-45d0-af42-21de225b61b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5002df604f7c489f98db223cbf8f997d', 'request_id': 'req-01de2218-fe94-45d0-af42-21de225b61b5'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:37.217219 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-01de2218-fe94-45d0-af42-21de225b61b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5002df604f7c489f98db223cbf8f997d', 'request_id': 'req-01de2218-fe94-45d0-af42-21de225b61b5'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:37.300925 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-01de2218-fe94-45d0-af42-21de225b61b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5002df604f7c489f98db223cbf8f997d', 'request_id': 'req-01de2218-fe94-45d0-af42-21de225b61b5'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:37.317949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-01de2218-fe94-45d0-af42-21de225b61b5 admin admin] The token cache is being invalidate because project 5002df604f7c489f98db223cbf8f997d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:52:37.318394 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-01de2218-fe94-45d0-af42-21de225b61b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:37.319126 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-01de2218-fe94-45d0-af42-21de225b61b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:37.320788 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1175/2370] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:52:37 2022] DELETE /identity/v3/projects/5002df604f7c489f98db223cbf8f997d => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:52:37.324879 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2a3d8-b51f-43a4-9d92-8ee07207137c service neutron] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:37.336425 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2a3d8-b51f-43a4-9d92-8ee07207137c service neutron] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:37.359756 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:52:37.420094 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a7d9f9-82bf-44f7-b62d-9b6587d5ced1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:37.420849 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a7d9f9-82bf-44f7-b62d-9b6587d5ced1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:37.421461 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a7d9f9-82bf-44f7-b62d-9b6587d5ced1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:37.421461 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a7d9f9-82bf-44f7-b62d-9b6587d5ced1 admin admin] PATH_INFO: `/v3/users/58bda88187a043038ef8b4be93a7d95d` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:37.422936 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a7d9f9-82bf-44f7-b62d-9b6587d5ced1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58bda88187a043038ef8b4be93a7d95d)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:37.425880 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a7d9f9-82bf-44f7-b62d-9b6587d5ced1 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:37.790398 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-36a7d9f9-82bf-44f7-b62d-9b6587d5ced1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58bda88187a043038ef8b4be93a7d95d'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:37.797864 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-36a7d9f9-82bf-44f7-b62d-9b6587d5ced1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58bda88187a043038ef8b4be93a7d95d'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:37.813860 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-36a7d9f9-82bf-44f7-b62d-9b6587d5ced1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58bda88187a043038ef8b4be93a7d95d'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:37.814681 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-36a7d9f9-82bf-44f7-b62d-9b6587d5ced1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58bda88187a043038ef8b4be93a7d95d'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:37.889581 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-36a7d9f9-82bf-44f7-b62d-9b6587d5ced1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58bda88187a043038ef8b4be93a7d95d'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:37.898147 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1196/2371] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:52:37 2022] DELETE /identity/v3/users/58bda88187a043038ef8b4be93a7d95d => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:52:38.486195 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de2218-fe94-45d0-af42-21de225b61b5 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:38.488715 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-01de2218-fe94-45d0-af42-21de225b61b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:38.519170 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de2218-fe94-45d0-af42-21de225b61b5 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:38.615882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbcd078-c4f1-44a5-b8ea-e9f98d2a20fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:38.618054 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbcd078-c4f1-44a5-b8ea-e9f98d2a20fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:38.618054 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbcd078-c4f1-44a5-b8ea-e9f98d2a20fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:38.618054 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbcd078-c4f1-44a5-b8ea-e9f98d2a20fb admin admin] PATH_INFO: `/v3/projects/0dcfe89166a74fe58c5bbebc676d58f9` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:38.619020 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbcd078-c4f1-44a5-b8ea-e9f98d2a20fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dcfe89166a74fe58c5bbebc676d58f9)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:38.621531 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbcd078-c4f1-44a5-b8ea-e9f98d2a20fb admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:38.647800 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:52:38.758304 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7cbcd078-c4f1-44a5-b8ea-e9f98d2a20fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dcfe89166a74fe58c5bbebc676d58f9', 'request_id': 'req-7cbcd078-c4f1-44a5-b8ea-e9f98d2a20fb'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:38.767704 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7cbcd078-c4f1-44a5-b8ea-e9f98d2a20fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dcfe89166a74fe58c5bbebc676d58f9', 'request_id': 'req-7cbcd078-c4f1-44a5-b8ea-e9f98d2a20fb'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:38.838063 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18113c3-e9c7-4a3a-9103-e6aba97f49e3 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:38.838484 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18113c3-e9c7-4a3a-9103-e6aba97f49e3 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:38.838617 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18113c3-e9c7-4a3a-9103-e6aba97f49e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:38.859617 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-a18113c3-e9c7-4a3a-9103-e6aba97f49e3 None None] MFA Rules not processed for user `7ffffa27986f461ebc10c23c3c9817f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:52:38.862015 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-a18113c3-e9c7-4a3a-9103-e6aba97f49e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:38.877331 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7cbcd078-c4f1-44a5-b8ea-e9f98d2a20fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dcfe89166a74fe58c5bbebc676d58f9', 'request_id': 'req-7cbcd078-c4f1-44a5-b8ea-e9f98d2a20fb'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:38.889885 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7cbcd078-c4f1-44a5-b8ea-e9f98d2a20fb admin admin] The token cache is being invalidate because project 0dcfe89166a74fe58c5bbebc676d58f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:52:38.890205 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7cbcd078-c4f1-44a5-b8ea-e9f98d2a20fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:38.890854 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-7cbcd078-c4f1-44a5-b8ea-e9f98d2a20fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:38.892538 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1176/2372] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:52:38 2022] DELETE /identity/v3/projects/0dcfe89166a74fe58c5bbebc676d58f9 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:52:38.897029 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbcd078-c4f1-44a5-b8ea-e9f98d2a20fb admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:38.898629 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-7cbcd078-c4f1-44a5-b8ea-e9f98d2a20fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:38.924127 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1197/2373] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:52:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:52:38.968258 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbcd078-c4f1-44a5-b8ea-e9f98d2a20fb admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:38.974827 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c95706-950b-47e2-8376-7bb63eb6c919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:38.975522 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c95706-950b-47e2-8376-7bb63eb6c919 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:38.975803 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c95706-950b-47e2-8376-7bb63eb6c919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:38.976074 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c95706-950b-47e2-8376-7bb63eb6c919 admin admin] PATH_INFO: `/v3/users/a53ea1bae1914a69a9e1f94c9ae29a6f` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:38.977243 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c95706-950b-47e2-8376-7bb63eb6c919 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a53ea1bae1914a69a9e1f94c9ae29a6f)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:38.979276 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c95706-950b-47e2-8376-7bb63eb6c919 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:39.028576 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18113c3-e9c7-4a3a-9103-e6aba97f49e3 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:39.030243 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-a18113c3-e9c7-4a3a-9103-e6aba97f49e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:39.047048 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18113c3-e9c7-4a3a-9103-e6aba97f49e3 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:39.053026 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d01572-6338-4fd1-87f6-19ec1328feac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:39.053682 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d01572-6338-4fd1-87f6-19ec1328feac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:39.053959 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d01572-6338-4fd1-87f6-19ec1328feac admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:39.054237 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d01572-6338-4fd1-87f6-19ec1328feac admin admin] PATH_INFO: `/v3/users/78b322344df24e8ca513c19361b79257` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:39.055431 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d01572-6338-4fd1-87f6-19ec1328feac admin admin] RBAC: Authorizing `identity:delete_user(user_id=78b322344df24e8ca513c19361b79257)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:39.057368 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d01572-6338-4fd1-87f6-19ec1328feac admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:39.249047 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-24c95706-950b-47e2-8376-7bb63eb6c919 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a53ea1bae1914a69a9e1f94c9ae29a6f'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:39.259520 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-24c95706-950b-47e2-8376-7bb63eb6c919 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a53ea1bae1914a69a9e1f94c9ae29a6f'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:39.260474 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-24c95706-950b-47e2-8376-7bb63eb6c919 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a53ea1bae1914a69a9e1f94c9ae29a6f'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:39.268491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-24c95706-950b-47e2-8376-7bb63eb6c919 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a53ea1bae1914a69a9e1f94c9ae29a6f'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:39.301820 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:52:39.304228 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-02d01572-6338-4fd1-87f6-19ec1328feac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78b322344df24e8ca513c19361b79257'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:39.312539 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-02d01572-6338-4fd1-87f6-19ec1328feac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78b322344df24e8ca513c19361b79257'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:39.326219 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-02d01572-6338-4fd1-87f6-19ec1328feac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78b322344df24e8ca513c19361b79257'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:39.326969 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-02d01572-6338-4fd1-87f6-19ec1328feac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78b322344df24e8ca513c19361b79257'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:39.369114 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-24c95706-950b-47e2-8376-7bb63eb6c919 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a53ea1bae1914a69a9e1f94c9ae29a6f'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:39.383382 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1177/2374] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:52:38 2022] DELETE /identity/v3/users/a53ea1bae1914a69a9e1f94c9ae29a6f => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:52:39.437987 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-02d01572-6338-4fd1-87f6-19ec1328feac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78b322344df24e8ca513c19361b79257'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:39.445857 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1198/2375] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:52:39 2022] DELETE /identity/v3/users/78b322344df24e8ca513c19361b79257 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:52:39.869286 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c95706-950b-47e2-8376-7bb63eb6c919 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:39.869737 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-24c95706-950b-47e2-8376-7bb63eb6c919 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:39.907701 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:52:39.946943 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c95706-950b-47e2-8376-7bb63eb6c919 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:40.014176 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25724822-c362-4c3b-9523-a914ec169b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:40.014918 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25724822-c362-4c3b-9523-a914ec169b09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:40.015340 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25724822-c362-4c3b-9523-a914ec169b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:40.015673 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25724822-c362-4c3b-9523-a914ec169b09 admin admin] PATH_INFO: `/v3/projects/eb983cfa7dfe47b19a0a2d5ba652dabb` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:40.016977 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25724822-c362-4c3b-9523-a914ec169b09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb983cfa7dfe47b19a0a2d5ba652dabb)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:40.020361 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25724822-c362-4c3b-9523-a914ec169b09 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:40.146196 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-25724822-c362-4c3b-9523-a914ec169b09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb983cfa7dfe47b19a0a2d5ba652dabb', 'request_id': 'req-25724822-c362-4c3b-9523-a914ec169b09'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:40.151574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-25724822-c362-4c3b-9523-a914ec169b09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb983cfa7dfe47b19a0a2d5ba652dabb', 'request_id': 'req-25724822-c362-4c3b-9523-a914ec169b09'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:40.169015 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d01572-6338-4fd1-87f6-19ec1328feac admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:40.170510 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-02d01572-6338-4fd1-87f6-19ec1328feac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:40.185463 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d01572-6338-4fd1-87f6-19ec1328feac admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:40.247783 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a989c-9f10-4404-a55d-ddf4b1f5b108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:40.248427 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a989c-9f10-4404-a55d-ddf4b1f5b108 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:40.248706 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a989c-9f10-4404-a55d-ddf4b1f5b108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:40.248980 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a989c-9f10-4404-a55d-ddf4b1f5b108 admin admin] PATH_INFO: `/v3/projects/aa9769292c00415399b0358aa260f920` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:40.250128 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106a989c-9f10-4404-a55d-ddf4b1f5b108 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa9769292c00415399b0358aa260f920)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:40.252094 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106a989c-9f10-4404-a55d-ddf4b1f5b108 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:40.254948 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-25724822-c362-4c3b-9523-a914ec169b09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb983cfa7dfe47b19a0a2d5ba652dabb', 'request_id': 'req-25724822-c362-4c3b-9523-a914ec169b09'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:40.285759 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-25724822-c362-4c3b-9523-a914ec169b09 admin admin] The token cache is being invalidate because project eb983cfa7dfe47b19a0a2d5ba652dabb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:52:40.285759 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-25724822-c362-4c3b-9523-a914ec169b09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:40.285759 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-25724822-c362-4c3b-9523-a914ec169b09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:40.285759 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1178/2376] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:52:39 2022] DELETE /identity/v3/projects/eb983cfa7dfe47b19a0a2d5ba652dabb => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:52:40.285759 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e55db-3364-4d09-8ea7-66ef3a621807 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:40.286857 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e55db-3364-4d09-8ea7-66ef3a621807 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:40.286857 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e55db-3364-4d09-8ea7-66ef3a621807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:40.302318 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-e11e55db-3364-4d09-8ea7-66ef3a621807 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:52:40.303586 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e11e55db-3364-4d09-8ea7-66ef3a621807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:40.311581 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1179/2377] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:52:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:52:40.318855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb5847-4258-4299-b47c-4626ca613a5f None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:40.319141 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb5847-4258-4299-b47c-4626ca613a5f None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:40.319456 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb5847-4258-4299-b47c-4626ca613a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:40.338231 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-f5bb5847-4258-4299-b47c-4626ca613a5f None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:52:40.339443 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-f5bb5847-4258-4299-b47c-4626ca613a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:40.348427 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1180/2378] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:52:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:52:40.352334 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bb5847-4258-4299-b47c-4626ca613a5f None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:40.362507 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bb5847-4258-4299-b47c-4626ca613a5f None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:40.370903 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f19fd8-dbfd-4107-8cda-d2b3bb9c5fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:40.372219 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f19fd8-dbfd-4107-8cda-d2b3bb9c5fcf admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:40.372612 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f19fd8-dbfd-4107-8cda-d2b3bb9c5fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:40.372884 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f19fd8-dbfd-4107-8cda-d2b3bb9c5fcf admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:40.373908 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f19fd8-dbfd-4107-8cda-d2b3bb9c5fcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:52:40.374268 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f19fd8-dbfd-4107-8cda-d2b3bb9c5fcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:40.375933 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f19fd8-dbfd-4107-8cda-d2b3bb9c5fcf admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:40.393484 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1181/2379] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:52:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:52:40.398079 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f19fd8-dbfd-4107-8cda-d2b3bb9c5fcf admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:40.406181 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-106a989c-9f10-4404-a55d-ddf4b1f5b108 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa9769292c00415399b0358aa260f920', 'request_id': 'req-106a989c-9f10-4404-a55d-ddf4b1f5b108'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:40.409276 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f19fd8-dbfd-4107-8cda-d2b3bb9c5fcf admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:40.413095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-106a989c-9f10-4404-a55d-ddf4b1f5b108 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa9769292c00415399b0358aa260f920', 'request_id': 'req-106a989c-9f10-4404-a55d-ddf4b1f5b108'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:40.471856 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc1812d-55d5-41a2-963c-529c7fa52c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:40.475126 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1812d-55d5-41a2-963c-529c7fa52c89 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:40.475126 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1812d-55d5-41a2-963c-529c7fa52c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:40.475126 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1812d-55d5-41a2-963c-529c7fa52c89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:40.475126 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc1812d-55d5-41a2-963c-529c7fa52c89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:40.476954 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:52:40.479655 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc1812d-55d5-41a2-963c-529c7fa52c89 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:40.483734 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-106a989c-9f10-4404-a55d-ddf4b1f5b108 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa9769292c00415399b0358aa260f920', 'request_id': 'req-106a989c-9f10-4404-a55d-ddf4b1f5b108'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:40.498672 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-106a989c-9f10-4404-a55d-ddf4b1f5b108 admin admin] The token cache is being invalidate because project aa9769292c00415399b0358aa260f920 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:52:40.499705 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-106a989c-9f10-4404-a55d-ddf4b1f5b108 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:40.499999 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-106a989c-9f10-4404-a55d-ddf4b1f5b108 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:40.501776 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1199/2380] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:52:40 2022] DELETE /identity/v3/projects/aa9769292c00415399b0358aa260f920 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:52:40.506525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48424d44-8ca0-4865-9fb3-eaedd2b475af None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:40.506860 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48424d44-8ca0-4865-9fb3-eaedd2b475af None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:40.507133 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48424d44-8ca0-4865-9fb3-eaedd2b475af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:40.529217 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-48424d44-8ca0-4865-9fb3-eaedd2b475af None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:52:40.530451 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-48424d44-8ca0-4865-9fb3-eaedd2b475af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:40.540588 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1200/2381] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:52:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:52:40.549797 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87122e0a-5f67-4787-948c-efd1655dd63d None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:40.550088 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87122e0a-5f67-4787-948c-efd1655dd63d None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:40.550360 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87122e0a-5f67-4787-948c-efd1655dd63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:40.573655 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1182/2382] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:52:40 2022] POST /identity/v3/projects => generated 367 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:52:40.575897 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc1812d-55d5-41a2-963c-529c7fa52c89 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:40.577030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-bfc1812d-55d5-41a2-963c-529c7fa52c89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:40.578823 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-87122e0a-5f67-4787-948c-efd1655dd63d None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:52:40.580046 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-87122e0a-5f67-4787-948c-efd1655dd63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:40.624640 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:52:40.644787 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1201/2383] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:52:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:52:40.648448 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc1812d-55d5-41a2-963c-529c7fa52c89 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:40.649142 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87122e0a-5f67-4787-948c-efd1655dd63d None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:40.653286 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68080d-62b6-40fe-8828-748bb8f1b0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:40.653903 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68080d-62b6-40fe-8828-748bb8f1b0ba admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:40.654185 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68080d-62b6-40fe-8828-748bb8f1b0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:40.662028 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68080d-62b6-40fe-8828-748bb8f1b0ba admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:40.662028 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c68080d-62b6-40fe-8828-748bb8f1b0ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:40.662028 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c68080d-62b6-40fe-8828-748bb8f1b0ba admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:40.662797 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87122e0a-5f67-4787-948c-efd1655dd63d None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:40.664120 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460746cc-b4ab-41bc-ba1f-42a92bd6da1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:40.664790 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460746cc-b4ab-41bc-ba1f-42a92bd6da1f admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:40.665240 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460746cc-b4ab-41bc-ba1f-42a92bd6da1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:40.665583 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460746cc-b4ab-41bc-ba1f-42a92bd6da1f admin admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:40.666266 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460746cc-b4ab-41bc-ba1f-42a92bd6da1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:52:40.666483 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460746cc-b4ab-41bc-ba1f-42a92bd6da1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:40.668183 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460746cc-b4ab-41bc-ba1f-42a92bd6da1f admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:40.683693 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1202/2384] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:52:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:52:40.688997 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460746cc-b4ab-41bc-ba1f-42a92bd6da1f admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:40.698688 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460746cc-b4ab-41bc-ba1f-42a92bd6da1f admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:40.703479 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ccd72e-1a06-46aa-ac8c-a6a844579cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:40.704079 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ccd72e-1a06-46aa-ac8c-a6a844579cbd admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:40.704363 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ccd72e-1a06-46aa-ac8c-a6a844579cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:40.704639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ccd72e-1a06-46aa-ac8c-a6a844579cbd admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:40.705261 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ccd72e-1a06-46aa-ac8c-a6a844579cbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:40.707740 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ccd72e-1a06-46aa-ac8c-a6a844579cbd admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:40.793612 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1183/2385] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:52:40 2022] POST /identity/v3/users => generated 284 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:52:40.793806 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1203/2386] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:52:40 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:52:40.794608 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68080d-62b6-40fe-8828-748bb8f1b0ba admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:40.796198 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ccd72e-1a06-46aa-ac8c-a6a844579cbd admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:40.809652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68080d-62b6-40fe-8828-748bb8f1b0ba admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:40.879801 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de70ed96-bf9a-44b4-bfb8-f0ef6869b20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:40.880523 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70ed96-bf9a-44b4-bfb8-f0ef6869b20d admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:40.880695 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70ed96-bf9a-44b4-bfb8-f0ef6869b20d admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:40.880968 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70ed96-bf9a-44b4-bfb8-f0ef6869b20d admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:40.881642 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de70ed96-bf9a-44b4-bfb8-f0ef6869b20d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:40.882381 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ccd72e-1a06-46aa-ac8c-a6a844579cbd admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:40.883268 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de70ed96-bf9a-44b4-bfb8-f0ef6869b20d admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:40.887887 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d01a12e-bb76-4cd3-bc90-fbfb6c5ce17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:40.889018 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01a12e-bb76-4cd3-bc90-fbfb6c5ce17c admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:40.889391 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01a12e-bb76-4cd3-bc90-fbfb6c5ce17c admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:40.889734 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01a12e-bb76-4cd3-bc90-fbfb6c5ce17c admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:40.890428 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d01a12e-bb76-4cd3-bc90-fbfb6c5ce17c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:40.892446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d01a12e-bb76-4cd3-bc90-fbfb6c5ce17c admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:40.895786 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1184/2387] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 12:52:40.900319 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de70ed96-bf9a-44b4-bfb8-f0ef6869b20d admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:40.912628 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de70ed96-bf9a-44b4-bfb8-f0ef6869b20d admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:40.920235 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7261e0-59c7-49ae-937d-98a99d9f2c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:40.925319 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7261e0-59c7-49ae-937d-98a99d9f2c39 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:40.925319 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7261e0-59c7-49ae-937d-98a99d9f2c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:40.925319 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7261e0-59c7-49ae-937d-98a99d9f2c39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:40.925319 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7261e0-59c7-49ae-937d-98a99d9f2c39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:40.925319 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7261e0-59c7-49ae-937d-98a99d9f2c39 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:40.938443 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1185/2388] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 12:52:40.940320 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7261e0-59c7-49ae-937d-98a99d9f2c39 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:40.950926 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7261e0-59c7-49ae-937d-98a99d9f2c39 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:40.955559 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c426ed-6bc0-49b2-8424-2e116d0f8d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:40.956210 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c426ed-6bc0-49b2-8424-2e116d0f8d9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:40.956486 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c426ed-6bc0-49b2-8424-2e116d0f8d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:40.956821 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c426ed-6bc0-49b2-8424-2e116d0f8d9b admin admin] PATH_INFO: `/v3/projects/592727cea0f34a3b804f4deb42ead98c/users/2862518a747f475c8d4db7681a29a7a0/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:40.972937 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c426ed-6bc0-49b2-8424-2e116d0f8d9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=592727cea0f34a3b804f4deb42ead98c, user_id=2862518a747f475c8d4db7681a29a7a0, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:40.975690 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c426ed-6bc0-49b2-8424-2e116d0f8d9b admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:41.062866 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1204/2389] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:52:40 2022] POST /identity/v3/users => generated 288 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:52:41.067050 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d01a12e-bb76-4cd3-bc90-fbfb6c5ce17c admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:41.076860 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d01a12e-bb76-4cd3-bc90-fbfb6c5ce17c admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:41.081460 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3eae42-c60c-4b55-a2c1-c7263382ea42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:41.082055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3eae42-c60c-4b55-a2c1-c7263382ea42 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:41.082332 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3eae42-c60c-4b55-a2c1-c7263382ea42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:41.082750 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3eae42-c60c-4b55-a2c1-c7263382ea42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:41.083237 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3eae42-c60c-4b55-a2c1-c7263382ea42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:41.084851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3eae42-c60c-4b55-a2c1-c7263382ea42 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:41.088054 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1186/2390] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:52:40 2022] PUT /identity/v3/projects/592727cea0f34a3b804f4deb42ead98c/users/2862518a747f475c8d4db7681a29a7a0/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:52:41.099422 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1205/2391] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 12:52:41.102628 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c426ed-6bc0-49b2-8424-2e116d0f8d9b admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:41.109285 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f533a80-e80a-4bc1-903f-3bf727360730 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:41.109576 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f533a80-e80a-4bc1-903f-3bf727360730 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:41.109901 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f533a80-e80a-4bc1-903f-3bf727360730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:41.122868 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:52:41.144742 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-8f533a80-e80a-4bc1-903f-3bf727360730 None None] MFA Rules not processed for user `2862518a747f475c8d4db7681a29a7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:52:41.145955 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-8f533a80-e80a-4bc1-903f-3bf727360730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:41.191999 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c426ed-6bc0-49b2-8424-2e116d0f8d9b admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:41.199537 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c883ba-bd62-44b6-894a-5baf6ff550f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:41.200470 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c883ba-bd62-44b6-894a-5baf6ff550f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:41.200980 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c883ba-bd62-44b6-894a-5baf6ff550f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:41.201452 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c883ba-bd62-44b6-894a-5baf6ff550f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:41.202298 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c883ba-bd62-44b6-894a-5baf6ff550f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:41.204119 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c883ba-bd62-44b6-894a-5baf6ff550f5 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:41.220109 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1187/2392] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 12:52:41.223031 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c883ba-bd62-44b6-894a-5baf6ff550f5 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:41.239476 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c883ba-bd62-44b6-894a-5baf6ff550f5 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:41.256197 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25070652-4e5f-44ea-89f1-943e185c3ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:41.257038 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25070652-4e5f-44ea-89f1-943e185c3ecc admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:41.257417 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25070652-4e5f-44ea-89f1-943e185c3ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:41.257918 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25070652-4e5f-44ea-89f1-943e185c3ecc admin admin] PATH_INFO: `/v3/projects/49280b136d6f4712bbf21c9a88f30764/users/89599062cda64ece827defdac3b05ea2/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:41.277549 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1206/2393] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:52:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:52:41.281901 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25070652-4e5f-44ea-89f1-943e185c3ecc admin admin] RBAC: Authorizing `identity:create_grant(project_id=49280b136d6f4712bbf21c9a88f30764, user_id=89599062cda64ece827defdac3b05ea2, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:41.292838 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25070652-4e5f-44ea-89f1-943e185c3ecc admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:41.293479 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f533a80-e80a-4bc1-903f-3bf727360730 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:41.294545 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-8f533a80-e80a-4bc1-903f-3bf727360730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:41.317695 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:52:41.370674 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f533a80-e80a-4bc1-903f-3bf727360730 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:41.374511 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de153482-0300-45eb-b10f-0755e0685fea service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b697e4847b13496d98a543675f5df1c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:41.375129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de153482-0300-45eb-b10f-0755e0685fea service glance] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:41.375442 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de153482-0300-45eb-b10f-0755e0685fea service glance] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:41.375718 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de153482-0300-45eb-b10f-0755e0685fea service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:41.384269 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de153482-0300-45eb-b10f-0755e0685fea service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:41.385687 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de153482-0300-45eb-b10f-0755e0685fea service glance] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:41.401555 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1188/2394] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:52:41 2022] PUT /identity/v3/projects/49280b136d6f4712bbf21c9a88f30764/users/89599062cda64ece827defdac3b05ea2/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:52:41.403902 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826e65f-1826-4350-9a41-c644edef1eca None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:41.404055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1207/2395] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:52:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:52:41.404165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826e65f-1826-4350-9a41-c644edef1eca None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:41.404393 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826e65f-1826-4350-9a41-c644edef1eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:41.438019 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-4826e65f-1826-4350-9a41-c644edef1eca None None] MFA Rules not processed for user `89599062cda64ece827defdac3b05ea2`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:52:41.439733 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4826e65f-1826-4350-9a41-c644edef1eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:41.641521 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1189/2396] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:52:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:52:41.653999 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de153482-0300-45eb-b10f-0755e0685fea service glance] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:41.655537 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-de153482-0300-45eb-b10f-0755e0685fea service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:41.721978 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de153482-0300-45eb-b10f-0755e0685fea service glance] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:41.774396 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a67bde-ddf8-4a7e-b50d-a4e8fdb8c910 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:41.775003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a67bde-ddf8-4a7e-b50d-a4e8fdb8c910 service cinder] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:41.775284 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a67bde-ddf8-4a7e-b50d-a4e8fdb8c910 service cinder] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:41.775581 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a67bde-ddf8-4a7e-b50d-a4e8fdb8c910 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:41.783928 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a67bde-ddf8-4a7e-b50d-a4e8fdb8c910 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:41.791498 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a67bde-ddf8-4a7e-b50d-a4e8fdb8c910 service cinder] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:41.795206 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1208/2397] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:52:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:52:42.011886 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826e65f-1826-4350-9a41-c644edef1eca None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:42.014143 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4826e65f-1826-4350-9a41-c644edef1eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:42.102125 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826e65f-1826-4350-9a41-c644edef1eca None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:42.107993 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b91507-07fb-4cbf-81e5-31b9ed3667fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25eef64e92e9422ea62508c3eb0a197e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:42.108687 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b91507-07fb-4cbf-81e5-31b9ed3667fd service placement] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:42.109023 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b91507-07fb-4cbf-81e5-31b9ed3667fd service placement] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:42.109372 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b91507-07fb-4cbf-81e5-31b9ed3667fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:42.111180 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-36b91507-07fb-4cbf-81e5-31b9ed3667fd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:42.124971 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b91507-07fb-4cbf-81e5-31b9ed3667fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:42.126817 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b91507-07fb-4cbf-81e5-31b9ed3667fd service placement] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:42.142808 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1190/2398] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:52:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:52:45.479944 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1929f2-51cb-498d-8135-9b799cee830a None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:45.480430 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1929f2-51cb-498d-8135-9b799cee830a None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:45.480554 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1929f2-51cb-498d-8135-9b799cee830a None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:45.482126 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1209/2399] 10.222.0.33 () {54 vars in 921 bytes} [Mon Oct 3 12:52:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:52:45.490722 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2e842-4e58-461a-917a-53eb9c05de60 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:45.491115 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2e842-4e58-461a-917a-53eb9c05de60 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:45.491284 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2e842-4e58-461a-917a-53eb9c05de60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:45.517026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:52:45.523480 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-e3d2e842-4e58-461a-917a-53eb9c05de60 None None] MFA Rules not processed for user `990f5837994f4a088c656c9b33e9a0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:52:45.524886 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e3d2e842-4e58-461a-917a-53eb9c05de60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:45.645531 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1191/2400] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Oct 3 12:52:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:52:50.623346 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1929f2-51cb-498d-8135-9b799cee830a None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:50.626070 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-1b1929f2-51cb-498d-8135-9b799cee830a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:50.693349 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1929f2-51cb-498d-8135-9b799cee830a None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:50.696902 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10838804-4104-42d8-83ab-ad7bcd79d3e7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b697e4847b13496d98a543675f5df1c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:50.697505 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10838804-4104-42d8-83ab-ad7bcd79d3e7 service glance] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:50.697789 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10838804-4104-42d8-83ab-ad7bcd79d3e7 service glance] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:50.698059 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10838804-4104-42d8-83ab-ad7bcd79d3e7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:50.708095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10838804-4104-42d8-83ab-ad7bcd79d3e7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:50.709508 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10838804-4104-42d8-83ab-ad7bcd79d3e7 service glance] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:50.719968 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1210/2401] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:52:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:52:50.897123 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03720e02-36b8-46ec-af6c-2bd0a30be573 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:50.897678 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03720e02-36b8-46ec-af6c-2bd0a30be573 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:50.897811 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03720e02-36b8-46ec-af6c-2bd0a30be573 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:50.899075 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1192/2402] 10.222.0.33 () {54 vars in 921 bytes} [Mon Oct 3 12:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:52:50.904537 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155151d4-4e6e-4369-899a-2cd92570dff2 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:50.904942 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155151d4-4e6e-4369-899a-2cd92570dff2 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:50.905272 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155151d4-4e6e-4369-899a-2cd92570dff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:50.928552 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-155151d4-4e6e-4369-899a-2cd92570dff2 None None] MFA Rules not processed for user `eb327a8a18ca414ab8b4111a4d13aad8`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:52:50.930408 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-155151d4-4e6e-4369-899a-2cd92570dff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:50.947379 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:52:50.989869 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1211/2403] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Oct 3 12:52:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:52:51.002175 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03720e02-36b8-46ec-af6c-2bd0a30be573 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:51.004074 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-03720e02-36b8-46ec-af6c-2bd0a30be573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:51.017342 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03720e02-36b8-46ec-af6c-2bd0a30be573 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:51.021458 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f4f6b-b64f-44a2-a45c-1b2533c142ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:51.022055 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f4f6b-b64f-44a2-a45c-1b2533c142ad service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:51.022329 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f4f6b-b64f-44a2-a45c-1b2533c142ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:51.022598 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f4f6b-b64f-44a2-a45c-1b2533c142ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:51.029583 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58f4f6b-b64f-44a2-a45c-1b2533c142ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:51.030995 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58f4f6b-b64f-44a2-a45c-1b2533c142ad service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:51.039354 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1193/2404] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:52:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:52:51.127940 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c2e55-2e2d-4de2-b5a3-625a768aa627 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:51.131095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c2e55-2e2d-4de2-b5a3-625a768aa627 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:51.131095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c2e55-2e2d-4de2-b5a3-625a768aa627 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:51.131095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1212/2405] 10.222.0.33 () {54 vars in 921 bytes} [Mon Oct 3 12:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:52:51.135048 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61126e7e-8352-49f4-869b-c556f50adf42 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:51.135356 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61126e7e-8352-49f4-869b-c556f50adf42 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:51.135632 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61126e7e-8352-49f4-869b-c556f50adf42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:51.155377 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-61126e7e-8352-49f4-869b-c556f50adf42 None None] MFA Rules not processed for user `990f5837994f4a088c656c9b33e9a0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:52:51.156558 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-61126e7e-8352-49f4-869b-c556f50adf42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:51.166120 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1194/2406] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Oct 3 12:52:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:52:51.176528 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c2e55-2e2d-4de2-b5a3-625a768aa627 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:51.188535 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c2e55-2e2d-4de2-b5a3-625a768aa627 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:51.192381 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9296492-280d-4f92-9b7f-ab52ff629977 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:51.193012 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9296492-280d-4f92-9b7f-ab52ff629977 service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:51.193289 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9296492-280d-4f92-9b7f-ab52ff629977 service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:51.193557 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9296492-280d-4f92-9b7f-ab52ff629977 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:51.205606 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9296492-280d-4f92-9b7f-ab52ff629977 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:51.205606 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9296492-280d-4f92-9b7f-ab52ff629977 service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:51.217939 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1213/2407] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:52:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:52:54.164722 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61126e7e-8352-49f4-869b-c556f50adf42 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:54.247682 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61126e7e-8352-49f4-869b-c556f50adf42 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:54.259195 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01bd61b-2fe1-402a-ae63-06a13f5bf1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:54.260143 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01bd61b-2fe1-402a-ae63-06a13f5bf1f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:54.260423 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01bd61b-2fe1-402a-ae63-06a13f5bf1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:54.260714 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01bd61b-2fe1-402a-ae63-06a13f5bf1f6 admin admin] PATH_INFO: `/v3/users/89599062cda64ece827defdac3b05ea2` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:54.261961 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01bd61b-2fe1-402a-ae63-06a13f5bf1f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89599062cda64ece827defdac3b05ea2)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:54.263958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01bd61b-2fe1-402a-ae63-06a13f5bf1f6 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:54.479283 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-f01bd61b-2fe1-402a-ae63-06a13f5bf1f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89599062cda64ece827defdac3b05ea2'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:54.497932 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-f01bd61b-2fe1-402a-ae63-06a13f5bf1f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89599062cda64ece827defdac3b05ea2'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:54.498644 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-f01bd61b-2fe1-402a-ae63-06a13f5bf1f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89599062cda64ece827defdac3b05ea2'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:54.506031 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-f01bd61b-2fe1-402a-ae63-06a13f5bf1f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89599062cda64ece827defdac3b05ea2'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:54.572649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-f01bd61b-2fe1-402a-ae63-06a13f5bf1f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89599062cda64ece827defdac3b05ea2'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:54.574232 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:52:54.579401 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1195/2408] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:52:54 2022] DELETE /identity/v3/users/89599062cda64ece827defdac3b05ea2 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:52:54.584235 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c80ae9-3302-4423-8b06-0c3e9a83c5a3 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:54.584706 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c80ae9-3302-4423-8b06-0c3e9a83c5a3 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:54.584804 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c80ae9-3302-4423-8b06-0c3e9a83c5a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:54.606119 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-b3c80ae9-3302-4423-8b06-0c3e9a83c5a3 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:52:54.607636 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-b3c80ae9-3302-4423-8b06-0c3e9a83c5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:54.673743 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1214/2409] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:52:54.694274 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01bd61b-2fe1-402a-ae63-06a13f5bf1f6 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:54.695743 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-f01bd61b-2fe1-402a-ae63-06a13f5bf1f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:54.759625 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01bd61b-2fe1-402a-ae63-06a13f5bf1f6 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:54.763333 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7c2d6c-c0c6-4781-8eb9-0c06c22179f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:54.763956 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c2d6c-c0c6-4781-8eb9-0c06c22179f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:54.764231 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c2d6c-c0c6-4781-8eb9-0c06c22179f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:54.764499 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c2d6c-c0c6-4781-8eb9-0c06c22179f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:54.774711 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7c2d6c-c0c6-4781-8eb9-0c06c22179f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:54.776154 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7c2d6c-c0c6-4781-8eb9-0c06c22179f6 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:54.786569 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1196/2410] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:52:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:52:55.227733 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c80ae9-3302-4423-8b06-0c3e9a83c5a3 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:55.229298 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-b3c80ae9-3302-4423-8b06-0c3e9a83c5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:55.242396 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c80ae9-3302-4423-8b06-0c3e9a83c5a3 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:55.247591 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5555cb0d-1c5c-4886-90c5-2d20ff55e6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:55.253720 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555cb0d-1c5c-4886-90c5-2d20ff55e6b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:55.253720 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555cb0d-1c5c-4886-90c5-2d20ff55e6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:55.253720 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555cb0d-1c5c-4886-90c5-2d20ff55e6b8 admin admin] PATH_INFO: `/v3/projects/49280b136d6f4712bbf21c9a88f30764` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:55.253720 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5555cb0d-1c5c-4886-90c5-2d20ff55e6b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49280b136d6f4712bbf21c9a88f30764)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:55.253720 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5555cb0d-1c5c-4886-90c5-2d20ff55e6b8 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:55.373462 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-5555cb0d-1c5c-4886-90c5-2d20ff55e6b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49280b136d6f4712bbf21c9a88f30764', 'request_id': 'req-5555cb0d-1c5c-4886-90c5-2d20ff55e6b8'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:55.382257 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-5555cb0d-1c5c-4886-90c5-2d20ff55e6b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49280b136d6f4712bbf21c9a88f30764', 'request_id': 'req-5555cb0d-1c5c-4886-90c5-2d20ff55e6b8'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:55.442987 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-5555cb0d-1c5c-4886-90c5-2d20ff55e6b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49280b136d6f4712bbf21c9a88f30764', 'request_id': 'req-5555cb0d-1c5c-4886-90c5-2d20ff55e6b8'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:55.460074 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-5555cb0d-1c5c-4886-90c5-2d20ff55e6b8 admin admin] The token cache is being invalidate because project 49280b136d6f4712bbf21c9a88f30764 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:52:55.465526 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-5555cb0d-1c5c-4886-90c5-2d20ff55e6b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:55.465526 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-5555cb0d-1c5c-4886-90c5-2d20ff55e6b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:52:55.465526 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1215/2411] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:52:55 2022] DELETE /identity/v3/projects/49280b136d6f4712bbf21c9a88f30764 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:52:55.469492 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1051354-dea4-48ce-92d3-9b5500396d5c None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:55.470094 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1051354-dea4-48ce-92d3-9b5500396d5c None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:55.470094 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1051354-dea4-48ce-92d3-9b5500396d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:55.490359 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-e1051354-dea4-48ce-92d3-9b5500396d5c None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:52:55.492061 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e1051354-dea4-48ce-92d3-9b5500396d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:55.560649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1197/2412] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:52:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:52:55.628761 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36b179-d3b0-4584-99f9-ec7734f50bd5 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:55.629302 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36b179-d3b0-4584-99f9-ec7734f50bd5 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:55.629302 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36b179-d3b0-4584-99f9-ec7734f50bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:55.649157 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-5f36b179-d3b0-4584-99f9-ec7734f50bd5 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:52:55.650367 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-5f36b179-d3b0-4584-99f9-ec7734f50bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:55.658348 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1216/2413] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:52:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:52:55.663216 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1051354-dea4-48ce-92d3-9b5500396d5c None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:55.672984 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1051354-dea4-48ce-92d3-9b5500396d5c None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:55.677952 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fd2d96-618a-4828-be28-2e0963966aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:55.678563 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd2d96-618a-4828-be28-2e0963966aca admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:55.678842 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd2d96-618a-4828-be28-2e0963966aca admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:55.679116 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd2d96-618a-4828-be28-2e0963966aca admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:55.679803 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fd2d96-618a-4828-be28-2e0963966aca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:52:55.680106 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fd2d96-618a-4828-be28-2e0963966aca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:55.681774 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fd2d96-618a-4828-be28-2e0963966aca admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:55.694726 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1198/2414] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:52:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:52:55.699387 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f36b179-d3b0-4584-99f9-ec7734f50bd5 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:55.709527 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f36b179-d3b0-4584-99f9-ec7734f50bd5 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:55.714512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec65eafb-339e-4564-8ee1-8c0cf81706cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:55.715436 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec65eafb-339e-4564-8ee1-8c0cf81706cc admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:55.715837 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec65eafb-339e-4564-8ee1-8c0cf81706cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:55.716210 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec65eafb-339e-4564-8ee1-8c0cf81706cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:55.716972 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec65eafb-339e-4564-8ee1-8c0cf81706cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:55.719033 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec65eafb-339e-4564-8ee1-8c0cf81706cc admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:55.832600 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1217/2415] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:52:55 2022] POST /identity/v3/projects => generated 371 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:52:55.836688 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fd2d96-618a-4828-be28-2e0963966aca admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:55.846575 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fd2d96-618a-4828-be28-2e0963966aca admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:55.904975 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0cf514-c97b-4a17-9406-67f10bce5893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:55.905583 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0cf514-c97b-4a17-9406-67f10bce5893 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:55.905864 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0cf514-c97b-4a17-9406-67f10bce5893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:55.906138 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0cf514-c97b-4a17-9406-67f10bce5893 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:55.906776 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0cf514-c97b-4a17-9406-67f10bce5893 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:55.908735 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0cf514-c97b-4a17-9406-67f10bce5893 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:55.996082 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1199/2416] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:52:55 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:52:56.000315 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec65eafb-339e-4564-8ee1-8c0cf81706cc admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:56.011942 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec65eafb-339e-4564-8ee1-8c0cf81706cc admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:56.016797 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e360d22b-f37d-4756-bc01-bb61ebe7b879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:56.017401 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e360d22b-f37d-4756-bc01-bb61ebe7b879 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:56.017724 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e360d22b-f37d-4756-bc01-bb61ebe7b879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:56.017944 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e360d22b-f37d-4756-bc01-bb61ebe7b879 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:56.018584 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e360d22b-f37d-4756-bc01-bb61ebe7b879 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:56.020911 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e360d22b-f37d-4756-bc01-bb61ebe7b879 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:56.034325 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1218/2417] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:52:56.036095 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0cf514-c97b-4a17-9406-67f10bce5893 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:56.046772 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0cf514-c97b-4a17-9406-67f10bce5893 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:56.051887 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da73af46-bb7d-4011-acae-5e07770a245d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:56.052480 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da73af46-bb7d-4011-acae-5e07770a245d admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:56.052759 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da73af46-bb7d-4011-acae-5e07770a245d admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:56.053028 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da73af46-bb7d-4011-acae-5e07770a245d admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:56.053779 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da73af46-bb7d-4011-acae-5e07770a245d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:56.055517 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da73af46-bb7d-4011-acae-5e07770a245d admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:56.066971 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1200/2418] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:52:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:52:56.071052 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e360d22b-f37d-4756-bc01-bb61ebe7b879 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:56.076345 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:52:56.082210 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e360d22b-f37d-4756-bc01-bb61ebe7b879 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:56.090572 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d610ac6a-9522-490c-88b2-c591753bd4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:56.090572 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610ac6a-9522-490c-88b2-c591753bd4e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:56.090572 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610ac6a-9522-490c-88b2-c591753bd4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:56.090572 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610ac6a-9522-490c-88b2-c591753bd4e5 admin admin] PATH_INFO: `/v3/projects/06a284bf10ff47d6bc58de80f92c164a/users/bafb4a06c49d47cdb970e3f9e08a2543/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:56.102543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d610ac6a-9522-490c-88b2-c591753bd4e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06a284bf10ff47d6bc58de80f92c164a, user_id=bafb4a06c49d47cdb970e3f9e08a2543, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:56.105314 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d610ac6a-9522-490c-88b2-c591753bd4e5 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:56.176340 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1219/2419] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:52:56 2022] PUT /identity/v3/projects/06a284bf10ff47d6bc58de80f92c164a/users/bafb4a06c49d47cdb970e3f9e08a2543/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:52:56.178602 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7019ab-7c60-4e67-934a-e9c7b2a0062a None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:56.178939 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7019ab-7c60-4e67-934a-e9c7b2a0062a None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:56.179276 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7019ab-7c60-4e67-934a-e9c7b2a0062a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:56.193344 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:52:56.200287 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-0f7019ab-7c60-4e67-934a-e9c7b2a0062a None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:52:56.201478 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-0f7019ab-7c60-4e67-934a-e9c7b2a0062a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:56.208205 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59a66a-5c8a-4d65-ac62-9fa02b78b3a7 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:56.208578 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59a66a-5c8a-4d65-ac62-9fa02b78b3a7 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:56.208932 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59a66a-5c8a-4d65-ac62-9fa02b78b3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:56.242426 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-1c59a66a-5c8a-4d65-ac62-9fa02b78b3a7 None None] MFA Rules not processed for user `2e6e4387a7c24bbc82c22eefe2a3d138`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:52:56.245655 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-1c59a66a-5c8a-4d65-ac62-9fa02b78b3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:56.298714 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1201/2420] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:52:56.320838 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7019ab-7c60-4e67-934a-e9c7b2a0062a None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:56.322291 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-0f7019ab-7c60-4e67-934a-e9c7b2a0062a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:56.351415 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1220/2421] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Oct 3 12:52:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:52:56.369859 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c59a66a-5c8a-4d65-ac62-9fa02b78b3a7 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:52:56.371517 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-1c59a66a-5c8a-4d65-ac62-9fa02b78b3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:52:56.397212 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7019ab-7c60-4e67-934a-e9c7b2a0062a None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:56.401121 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0190a2d3-5162-4771-9a9c-8fc8249fcc33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:56.401839 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190a2d3-5162-4771-9a9c-8fc8249fcc33 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:56.402191 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190a2d3-5162-4771-9a9c-8fc8249fcc33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:56.402532 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190a2d3-5162-4771-9a9c-8fc8249fcc33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:56.415588 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0190a2d3-5162-4771-9a9c-8fc8249fcc33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:56.417073 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0190a2d3-5162-4771-9a9c-8fc8249fcc33 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:56.430913 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1202/2422] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:52:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:52:56.457144 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c59a66a-5c8a-4d65-ac62-9fa02b78b3a7 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:52:56.466429 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e0a3f-ffcd-49bb-8d1a-5fb6406bcb6b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b111d72a34144073917de4f566a58233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:52:56.467666 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e0a3f-ffcd-49bb-8d1a-5fb6406bcb6b service swift] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:52:56.468139 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e0a3f-ffcd-49bb-8d1a-5fb6406bcb6b service swift] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:52:56.468636 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e0a3f-ffcd-49bb-8d1a-5fb6406bcb6b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:52:56.482094 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e0a3f-ffcd-49bb-8d1a-5fb6406bcb6b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:52:56.483832 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e0a3f-ffcd-49bb-8d1a-5fb6406bcb6b service swift] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:52:56.495134 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1221/2423] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Oct 3 12:52:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 03 12:53:04.625350 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b9e83-565a-48af-8da9-831c4297b7cb None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:53:04.625992 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b9e83-565a-48af-8da9-831c4297b7cb None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:53:04.625992 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b9e83-565a-48af-8da9-831c4297b7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:53:04.669675 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-8d7b9e83-565a-48af-8da9-831c4297b7cb None None] MFA Rules not processed for user `bafb4a06c49d47cdb970e3f9e08a2543`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:53:04.671058 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-8d7b9e83-565a-48af-8da9-831c4297b7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:53:04.835644 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1203/2424] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:53:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:53:04.838298 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e0a3f-ffcd-49bb-8d1a-5fb6406bcb6b service swift] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:53:04.875846 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e0a3f-ffcd-49bb-8d1a-5fb6406bcb6b service swift] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:53:04.891093 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca947b2f-7a65-406e-8b04-855ee3d212a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:53:04.891882 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca947b2f-7a65-406e-8b04-855ee3d212a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:53:04.892336 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca947b2f-7a65-406e-8b04-855ee3d212a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:53:04.892626 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca947b2f-7a65-406e-8b04-855ee3d212a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:53:04.893373 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca947b2f-7a65-406e-8b04-855ee3d212a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:53:04.900838 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca947b2f-7a65-406e-8b04-855ee3d212a3 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:53:05.149538 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1222/2425] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:53:04 2022] POST /identity/v3/projects => generated 373 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:53:05.160176 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7b9e83-565a-48af-8da9-831c4297b7cb None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:53:05.192186 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7b9e83-565a-48af-8da9-831c4297b7cb None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:53:05.370103 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:53:05.408055 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10735b4-3c56-4164-8aac-075e3b7b9494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:53:05.408469 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10735b4-3c56-4164-8aac-075e3b7b9494 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:53:05.408792 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10735b4-3c56-4164-8aac-075e3b7b9494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:53:05.409090 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10735b4-3c56-4164-8aac-075e3b7b9494 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:53:05.419940 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10735b4-3c56-4164-8aac-075e3b7b9494 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:53:05.423138 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10735b4-3c56-4164-8aac-075e3b7b9494 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:53:05.553928 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1204/2426] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:53:05 2022] POST /identity/v3/users => generated 293 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:53:05.560088 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca947b2f-7a65-406e-8b04-855ee3d212a3 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:53:05.582096 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca947b2f-7a65-406e-8b04-855ee3d212a3 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:53:05.592049 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d42b732-c384-4918-9d73-f9c8403b3ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:53:05.592818 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42b732-c384-4918-9d73-f9c8403b3ee7 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:53:05.593193 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42b732-c384-4918-9d73-f9c8403b3ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:53:05.593589 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42b732-c384-4918-9d73-f9c8403b3ee7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:53:05.594386 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d42b732-c384-4918-9d73-f9c8403b3ee7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:53:05.600234 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d42b732-c384-4918-9d73-f9c8403b3ee7 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:53:05.614281 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1223/2427] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:53:05 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:53:05.619107 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10735b4-3c56-4164-8aac-075e3b7b9494 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:53:05.637744 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10735b4-3c56-4164-8aac-075e3b7b9494 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:53:05.643036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e4136-0edd-42f7-9630-67920f8a718e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:53:05.644142 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e4136-0edd-42f7-9630-67920f8a718e admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:53:05.644523 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e4136-0edd-42f7-9630-67920f8a718e admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:53:05.644813 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e4136-0edd-42f7-9630-67920f8a718e admin admin] PATH_INFO: `/v3/projects/6ca39e772a0145148115322e8c815c59/users/5347c3430eb440f7916fbbeee37c1048/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:53:05.680685 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e4136-0edd-42f7-9630-67920f8a718e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ca39e772a0145148115322e8c815c59, user_id=5347c3430eb440f7916fbbeee37c1048, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:53:05.684485 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e4136-0edd-42f7-9630-67920f8a718e admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:53:05.843184 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1205/2428] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:53:05 2022] PUT /identity/v3/projects/6ca39e772a0145148115322e8c815c59/users/5347c3430eb440f7916fbbeee37c1048/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:53:05.846059 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d42b732-c384-4918-9d73-f9c8403b3ee7 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:53:05.860969 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d42b732-c384-4918-9d73-f9c8403b3ee7 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:53:05.932428 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:53:05.962557 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee311c8e-b94f-4098-9cb4-16a9c3afd7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:53:05.963183 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee311c8e-b94f-4098-9cb4-16a9c3afd7d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:53:05.963841 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee311c8e-b94f-4098-9cb4-16a9c3afd7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:53:05.964151 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee311c8e-b94f-4098-9cb4-16a9c3afd7d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:53:05.964790 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee311c8e-b94f-4098-9cb4-16a9c3afd7d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:53:05.966560 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee311c8e-b94f-4098-9cb4-16a9c3afd7d2 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:53:05.982482 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1224/2429] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:53:05 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 12:53:05.983708 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e4136-0edd-42f7-9630-67920f8a718e admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:53:05.999079 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e4136-0edd-42f7-9630-67920f8a718e admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:53:06.004063 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a49cf1-1a06-45aa-ac87-26b699bc864f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:53:06.004737 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a49cf1-1a06-45aa-ac87-26b699bc864f admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:53:06.004999 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a49cf1-1a06-45aa-ac87-26b699bc864f admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:53:06.005265 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a49cf1-1a06-45aa-ac87-26b699bc864f admin admin] PATH_INFO: `/v3/projects/6ca39e772a0145148115322e8c815c59/users/5347c3430eb440f7916fbbeee37c1048/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:53:06.007210 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a49cf1-1a06-45aa-ac87-26b699bc864f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ca39e772a0145148115322e8c815c59, user_id=5347c3430eb440f7916fbbeee37c1048, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:53:06.009910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a49cf1-1a06-45aa-ac87-26b699bc864f admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:53:06.025833 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1206/2430] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:53:05 2022] PUT /identity/v3/projects/6ca39e772a0145148115322e8c815c59/users/5347c3430eb440f7916fbbeee37c1048/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:53:06.394887 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee311c8e-b94f-4098-9cb4-16a9c3afd7d2 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:53:06.398719 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-ee311c8e-b94f-4098-9cb4-16a9c3afd7d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:53:06.414774 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee311c8e-b94f-4098-9cb4-16a9c3afd7d2 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:53:06.510300 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdac76e-abdd-4922-bc43-08db1137198a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:53:06.511529 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdac76e-abdd-4922-bc43-08db1137198a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:53:06.512135 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdac76e-abdd-4922-bc43-08db1137198a admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:53:06.514019 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdac76e-abdd-4922-bc43-08db1137198a admin admin] PATH_INFO: `/v3/users/b56aafd252e44345baeef1adb3f259ac` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:53:06.515540 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdac76e-abdd-4922-bc43-08db1137198a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b56aafd252e44345baeef1adb3f259ac)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:53:06.517823 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdac76e-abdd-4922-bc43-08db1137198a admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:53:06.726942 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-9bdac76e-abdd-4922-bc43-08db1137198a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b56aafd252e44345baeef1adb3f259ac'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:53:06.734746 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-9bdac76e-abdd-4922-bc43-08db1137198a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b56aafd252e44345baeef1adb3f259ac'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:53:06.748626 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-9bdac76e-abdd-4922-bc43-08db1137198a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b56aafd252e44345baeef1adb3f259ac'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:53:06.749349 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-9bdac76e-abdd-4922-bc43-08db1137198a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b56aafd252e44345baeef1adb3f259ac'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:53:06.825805 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-9bdac76e-abdd-4922-bc43-08db1137198a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b56aafd252e44345baeef1adb3f259ac'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:53:06.833265 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1225/2431] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:53:06 2022] DELETE /identity/v3/users/b56aafd252e44345baeef1adb3f259ac => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:53:06.838027 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0701c5-11db-4622-b9d0-36e9c4203be3 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:53:06.838496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0701c5-11db-4622-b9d0-36e9c4203be3 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:53:06.838588 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0701c5-11db-4622-b9d0-36e9c4203be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:53:06.859485 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-4b0701c5-11db-4622-b9d0-36e9c4203be3 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:53:06.862882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4b0701c5-11db-4622-b9d0-36e9c4203be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:53:06.946043 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1207/2432] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:53:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:53:06.954056 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdac76e-abdd-4922-bc43-08db1137198a admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:53:06.956206 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-9bdac76e-abdd-4922-bc43-08db1137198a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:53:07.024441 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdac76e-abdd-4922-bc43-08db1137198a admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:53:07.028426 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9364443f-ffd5-4727-95ce-7a3374b64990 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:53:07.029044 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9364443f-ffd5-4727-95ce-7a3374b64990 service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:53:07.029326 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9364443f-ffd5-4727-95ce-7a3374b64990 service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:53:07.029706 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9364443f-ffd5-4727-95ce-7a3374b64990 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:53:07.039481 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9364443f-ffd5-4727-95ce-7a3374b64990 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:53:07.041124 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9364443f-ffd5-4727-95ce-7a3374b64990 service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:53:07.046871 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:53:07.054661 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1226/2433] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:53:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:53:07.680146 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0701c5-11db-4622-b9d0-36e9c4203be3 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:53:07.681586 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4b0701c5-11db-4622-b9d0-36e9c4203be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:53:07.697600 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0701c5-11db-4622-b9d0-36e9c4203be3 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:53:07.702938 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8958bc-0f1f-4c3a-b6b0-9b5cbd094158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:53:07.703674 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8958bc-0f1f-4c3a-b6b0-9b5cbd094158 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:53:07.704114 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8958bc-0f1f-4c3a-b6b0-9b5cbd094158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:53:07.704388 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8958bc-0f1f-4c3a-b6b0-9b5cbd094158 admin admin] PATH_INFO: `/v3/projects/27ed41f5786a4a55922ed992f40c477e` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:53:07.705508 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8958bc-0f1f-4c3a-b6b0-9b5cbd094158 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27ed41f5786a4a55922ed992f40c477e)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:53:07.708152 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8958bc-0f1f-4c3a-b6b0-9b5cbd094158 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:53:07.839904 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-6f8958bc-0f1f-4c3a-b6b0-9b5cbd094158 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27ed41f5786a4a55922ed992f40c477e', 'request_id': 'req-6f8958bc-0f1f-4c3a-b6b0-9b5cbd094158'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:53:07.848178 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-6f8958bc-0f1f-4c3a-b6b0-9b5cbd094158 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27ed41f5786a4a55922ed992f40c477e', 'request_id': 'req-6f8958bc-0f1f-4c3a-b6b0-9b5cbd094158'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:53:07.928079 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-6f8958bc-0f1f-4c3a-b6b0-9b5cbd094158 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27ed41f5786a4a55922ed992f40c477e', 'request_id': 'req-6f8958bc-0f1f-4c3a-b6b0-9b5cbd094158'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:53:07.949976 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-6f8958bc-0f1f-4c3a-b6b0-9b5cbd094158 admin admin] The token cache is being invalidate because project 27ed41f5786a4a55922ed992f40c477e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:53:07.950384 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-6f8958bc-0f1f-4c3a-b6b0-9b5cbd094158 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:53:07.951154 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-6f8958bc-0f1f-4c3a-b6b0-9b5cbd094158 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:53:07.953337 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1208/2434] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:53:07 2022] DELETE /identity/v3/projects/27ed41f5786a4a55922ed992f40c477e => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:53:12.468834 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525204c-2af0-40ff-a6a0-200d9bab3527 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:53:12.469297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525204c-2af0-40ff-a6a0-200d9bab3527 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:53:12.469514 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525204c-2af0-40ff-a6a0-200d9bab3527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:53:12.505924 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-e525204c-2af0-40ff-a6a0-200d9bab3527 None None] MFA Rules not processed for user `5347c3430eb440f7916fbbeee37c1048`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:53:12.507074 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-e525204c-2af0-40ff-a6a0-200d9bab3527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:53:12.592215 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1227/2435] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:53:12 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:53:12.611504 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8958bc-0f1f-4c3a-b6b0-9b5cbd094158 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:53:12.613727 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-6f8958bc-0f1f-4c3a-b6b0-9b5cbd094158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:53:12.639061 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:53:12.714473 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8958bc-0f1f-4c3a-b6b0-9b5cbd094158 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:53:12.720751 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4eccd8-e05e-4216-b284-fa9468ee6c73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:53:12.722234 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4eccd8-e05e-4216-b284-fa9468ee6c73 service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:53:12.722729 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4eccd8-e05e-4216-b284-fa9468ee6c73 service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:53:12.723209 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4eccd8-e05e-4216-b284-fa9468ee6c73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:53:12.725135 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-9f4eccd8-e05e-4216-b284-fa9468ee6c73 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:53:12.816054 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4eccd8-e05e-4216-b284-fa9468ee6c73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:53:12.821537 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4eccd8-e05e-4216-b284-fa9468ee6c73 service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:53:12.832599 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1209/2436] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:53:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:53:13.525077 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e525204c-2af0-40ff-a6a0-200d9bab3527 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:53:13.526579 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-e525204c-2af0-40ff-a6a0-200d9bab3527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:53:13.641723 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e525204c-2af0-40ff-a6a0-200d9bab3527 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:53:13.647961 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83434850-d721-44e1-9d59-845322030016 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:53:13.649119 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83434850-d721-44e1-9d59-845322030016 service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:53:13.649476 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83434850-d721-44e1-9d59-845322030016 service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:53:13.649965 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83434850-d721-44e1-9d59-845322030016 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:53:13.652421 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-83434850-d721-44e1-9d59-845322030016 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:53:13.758001 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83434850-d721-44e1-9d59-845322030016 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:53:13.759529 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83434850-d721-44e1-9d59-845322030016 service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:53:13.770045 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1228/2437] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:53:13 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:53:16.312317 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec27aea-eac0-4dc5-aaf5-e8cf0467b11a None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:53:16.312737 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec27aea-eac0-4dc5-aaf5-e8cf0467b11a None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:53:16.312929 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec27aea-eac0-4dc5-aaf5-e8cf0467b11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:53:16.341734 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-1ec27aea-eac0-4dc5-aaf5-e8cf0467b11a None None] MFA Rules not processed for user `bafb4a06c49d47cdb970e3f9e08a2543`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:53:16.344177 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1ec27aea-eac0-4dc5-aaf5-e8cf0467b11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:53:16.352950 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1210/2438] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:53:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:53:23.465522 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699df36-deaf-494a-b656-cedf2225d17b None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:53:23.465991 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699df36-deaf-494a-b656-cedf2225d17b None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:53:23.466074 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699df36-deaf-494a-b656-cedf2225d17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:53:23.490075 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-8699df36-deaf-494a-b656-cedf2225d17b None None] MFA Rules not processed for user `2e6e4387a7c24bbc82c22eefe2a3d138`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:53:23.491564 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-8699df36-deaf-494a-b656-cedf2225d17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:53:23.525827 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:53:23.552086 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1229/2439] 10.222.0.33 () {58 vars in 967 bytes} [Mon Oct 3 12:53:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:53:23.576416 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec27aea-eac0-4dc5-aaf5-e8cf0467b11a None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:53:23.577852 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1ec27aea-eac0-4dc5-aaf5-e8cf0467b11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:53:23.669712 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec27aea-eac0-4dc5-aaf5-e8cf0467b11a None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:53:23.679744 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a80947b-b12a-4227-8a49-d2d1ee761c19 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b111d72a34144073917de4f566a58233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:53:23.680935 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80947b-b12a-4227-8a49-d2d1ee761c19 service swift] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:53:23.683012 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80947b-b12a-4227-8a49-d2d1ee761c19 service swift] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:53:23.683012 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80947b-b12a-4227-8a49-d2d1ee761c19 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:53:23.692288 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a80947b-b12a-4227-8a49-d2d1ee761c19 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:53:23.693920 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a80947b-b12a-4227-8a49-d2d1ee761c19 service swift] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:53:23.706033 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1211/2440] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Oct 3 12:53:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 12:53:24.789649 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8699df36-deaf-494a-b656-cedf2225d17b None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:53:24.791054 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-8699df36-deaf-494a-b656-cedf2225d17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:53:24.870585 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8699df36-deaf-494a-b656-cedf2225d17b None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:53:24.875184 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd1a7dc-093c-4c3f-a491-b4e69b1ad74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:53:24.875887 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1a7dc-093c-4c3f-a491-b4e69b1ad74e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:53:24.876174 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1a7dc-093c-4c3f-a491-b4e69b1ad74e admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:53:24.876450 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1a7dc-093c-4c3f-a491-b4e69b1ad74e admin admin] PATH_INFO: `/v3/users/2862518a747f475c8d4db7681a29a7a0` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:53:24.878722 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd1a7dc-093c-4c3f-a491-b4e69b1ad74e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2862518a747f475c8d4db7681a29a7a0)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:53:24.880719 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd1a7dc-093c-4c3f-a491-b4e69b1ad74e admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:53:25.096317 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-edd1a7dc-093c-4c3f-a491-b4e69b1ad74e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2862518a747f475c8d4db7681a29a7a0'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:53:25.104353 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-edd1a7dc-093c-4c3f-a491-b4e69b1ad74e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2862518a747f475c8d4db7681a29a7a0'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:53:25.118688 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-edd1a7dc-093c-4c3f-a491-b4e69b1ad74e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2862518a747f475c8d4db7681a29a7a0'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:53:25.119599 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-edd1a7dc-093c-4c3f-a491-b4e69b1ad74e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2862518a747f475c8d4db7681a29a7a0'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:53:25.202216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-edd1a7dc-093c-4c3f-a491-b4e69b1ad74e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2862518a747f475c8d4db7681a29a7a0'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:53:25.208792 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1230/2441] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:53:24 2022] DELETE /identity/v3/users/2862518a747f475c8d4db7681a29a7a0 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:53:25.213327 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1e1d4-d30c-4feb-8dc0-1c9c32a381b0 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:53:25.213805 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1e1d4-d30c-4feb-8dc0-1c9c32a381b0 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:53:25.213949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1e1d4-d30c-4feb-8dc0-1c9c32a381b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:53:25.235273 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-ead1e1d4-d30c-4feb-8dc0-1c9c32a381b0 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:53:25.237006 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ead1e1d4-d30c-4feb-8dc0-1c9c32a381b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:53:25.269648 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:53:25.304805 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1212/2442] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:53:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:53:25.311661 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd1a7dc-093c-4c3f-a491-b4e69b1ad74e admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:53:25.313085 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-edd1a7dc-093c-4c3f-a491-b4e69b1ad74e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:53:25.379172 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd1a7dc-093c-4c3f-a491-b4e69b1ad74e admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:53:25.383250 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fadccf-733c-4b53-b280-c96c8e12773b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:53:25.383878 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fadccf-733c-4b53-b280-c96c8e12773b service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:53:25.384518 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fadccf-733c-4b53-b280-c96c8e12773b service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:53:25.384792 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fadccf-733c-4b53-b280-c96c8e12773b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:53:25.394340 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fadccf-733c-4b53-b280-c96c8e12773b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:53:25.395809 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fadccf-733c-4b53-b280-c96c8e12773b service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:53:25.401189 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:53:25.407374 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1231/2443] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:53:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:53:25.890315 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead1e1d4-d30c-4feb-8dc0-1c9c32a381b0 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:53:25.891901 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ead1e1d4-d30c-4feb-8dc0-1c9c32a381b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:53:25.907520 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead1e1d4-d30c-4feb-8dc0-1c9c32a381b0 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:53:25.912840 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710eb20a-4757-49ca-844d-b2ce9b3853ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:53:25.915883 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710eb20a-4757-49ca-844d-b2ce9b3853ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:53:25.915883 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710eb20a-4757-49ca-844d-b2ce9b3853ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:53:25.915883 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710eb20a-4757-49ca-844d-b2ce9b3853ce admin admin] PATH_INFO: `/v3/projects/592727cea0f34a3b804f4deb42ead98c` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:53:25.915883 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710eb20a-4757-49ca-844d-b2ce9b3853ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=592727cea0f34a3b804f4deb42ead98c)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:53:25.917086 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710eb20a-4757-49ca-844d-b2ce9b3853ce admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:53:26.037794 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-710eb20a-4757-49ca-844d-b2ce9b3853ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '592727cea0f34a3b804f4deb42ead98c', 'request_id': 'req-710eb20a-4757-49ca-844d-b2ce9b3853ce'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:53:26.043615 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-710eb20a-4757-49ca-844d-b2ce9b3853ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '592727cea0f34a3b804f4deb42ead98c', 'request_id': 'req-710eb20a-4757-49ca-844d-b2ce9b3853ce'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:53:26.125642 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-710eb20a-4757-49ca-844d-b2ce9b3853ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '592727cea0f34a3b804f4deb42ead98c', 'request_id': 'req-710eb20a-4757-49ca-844d-b2ce9b3853ce'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:53:26.139853 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-710eb20a-4757-49ca-844d-b2ce9b3853ce admin admin] The token cache is being invalidate because project 592727cea0f34a3b804f4deb42ead98c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:53:26.140265 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-710eb20a-4757-49ca-844d-b2ce9b3853ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:53:26.140976 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-710eb20a-4757-49ca-844d-b2ce9b3853ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:53:26.143126 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1213/2444] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:53:25 2022] DELETE /identity/v3/projects/592727cea0f34a3b804f4deb42ead98c => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:53:26.148125 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5cdd4c-799d-439d-b680-0d671c37b035 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:53:26.148626 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5cdd4c-799d-439d-b680-0d671c37b035 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:53:26.148724 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5cdd4c-799d-439d-b680-0d671c37b035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:53:26.168151 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-1c5cdd4c-799d-439d-b680-0d671c37b035 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:53:26.169868 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-1c5cdd4c-799d-439d-b680-0d671c37b035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:53:26.235574 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1232/2445] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:53:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:53:26.245330 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1128f2-2ca0-4e89-b1ff-5bce7e47c892 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:53:26.245880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1128f2-2ca0-4e89-b1ff-5bce7e47c892 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:53:26.245880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1128f2-2ca0-4e89-b1ff-5bce7e47c892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:53:26.265495 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-1f1128f2-2ca0-4e89-b1ff-5bce7e47c892 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:53:26.266877 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1f1128f2-2ca0-4e89-b1ff-5bce7e47c892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:53:26.275999 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1214/2446] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:53:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:53:26.280921 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5cdd4c-799d-439d-b680-0d671c37b035 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:53:26.291544 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5cdd4c-799d-439d-b680-0d671c37b035 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:53:26.296180 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f4cc6-6acc-40bd-a30f-01cccd8abeeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:53:26.296781 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f4cc6-6acc-40bd-a30f-01cccd8abeeb admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:53:26.297060 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f4cc6-6acc-40bd-a30f-01cccd8abeeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:53:26.297331 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f4cc6-6acc-40bd-a30f-01cccd8abeeb admin admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:53:26.297967 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887f4cc6-6acc-40bd-a30f-01cccd8abeeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:53:26.298272 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887f4cc6-6acc-40bd-a30f-01cccd8abeeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:53:26.300106 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887f4cc6-6acc-40bd-a30f-01cccd8abeeb admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:53:26.312087 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1233/2447] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:53:26.317061 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1128f2-2ca0-4e89-b1ff-5bce7e47c892 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:53:26.326547 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1128f2-2ca0-4e89-b1ff-5bce7e47c892 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:53:26.331837 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a9903-d033-41f3-98b5-87c9096b7cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:53:26.332442 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a9903-d033-41f3-98b5-87c9096b7cc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:53:26.332721 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a9903-d033-41f3-98b5-87c9096b7cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:53:26.333017 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a9903-d033-41f3-98b5-87c9096b7cc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:53:26.333644 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196a9903-d033-41f3-98b5-87c9096b7cc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:53:26.335603 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196a9903-d033-41f3-98b5-87c9096b7cc8 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:53:26.424946 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1215/2448] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:53:26 2022] POST /identity/v3/projects => generated 399 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:53:26.429109 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f4cc6-6acc-40bd-a30f-01cccd8abeeb admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:53:26.438999 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f4cc6-6acc-40bd-a30f-01cccd8abeeb admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:53:26.495710 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d6af0-66ec-437b-a4bd-43b0106f31ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:53:26.496351 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d6af0-66ec-437b-a4bd-43b0106f31ab admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:53:26.496626 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d6af0-66ec-437b-a4bd-43b0106f31ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:53:26.496893 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d6af0-66ec-437b-a4bd-43b0106f31ab admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:53:26.497523 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d6af0-66ec-437b-a4bd-43b0106f31ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:53:26.499488 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d6af0-66ec-437b-a4bd-43b0106f31ab admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:53:26.580156 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1234/2449] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:53:26 2022] POST /identity/v3/users => generated 300 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:53:26.585279 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a9903-d033-41f3-98b5-87c9096b7cc8 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:53:26.598619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a9903-d033-41f3-98b5-87c9096b7cc8 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:53:26.603852 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71011f7c-fca1-48e7-bcd3-c5af41249355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:53:26.604466 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71011f7c-fca1-48e7-bcd3-c5af41249355 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:53:26.604743 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71011f7c-fca1-48e7-bcd3-c5af41249355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:53:26.605010 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71011f7c-fca1-48e7-bcd3-c5af41249355 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:53:26.605643 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71011f7c-fca1-48e7-bcd3-c5af41249355 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:53:26.607227 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71011f7c-fca1-48e7-bcd3-c5af41249355 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:53:26.618449 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1216/2450] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:53:26.622084 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d6af0-66ec-437b-a4bd-43b0106f31ab admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:53:26.632819 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d6af0-66ec-437b-a4bd-43b0106f31ab admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:53:26.637598 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f04032-8379-4c8c-ab64-aaefdffa682f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:53:26.638212 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f04032-8379-4c8c-ab64-aaefdffa682f admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:53:26.638488 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f04032-8379-4c8c-ab64-aaefdffa682f admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:53:26.638760 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f04032-8379-4c8c-ab64-aaefdffa682f admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:53:26.639443 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f04032-8379-4c8c-ab64-aaefdffa682f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:53:26.641066 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f04032-8379-4c8c-ab64-aaefdffa682f admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:53:26.652719 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1235/2451] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:53:26.656891 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71011f7c-fca1-48e7-bcd3-c5af41249355 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:53:26.667616 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71011f7c-fca1-48e7-bcd3-c5af41249355 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:53:26.672354 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d4c66-2d73-4c76-a7e8-2f4ae8348f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:53:26.673002 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d4c66-2d73-4c76-a7e8-2f4ae8348f12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:53:26.673284 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d4c66-2d73-4c76-a7e8-2f4ae8348f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:53:26.674091 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d4c66-2d73-4c76-a7e8-2f4ae8348f12 admin admin] PATH_INFO: `/v3/projects/b434dc51447f467b8c6521fdc66a2d49/users/8077ac2b5a214ceab9c7c9d6b1c32012/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:53:26.690144 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d4c66-2d73-4c76-a7e8-2f4ae8348f12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b434dc51447f467b8c6521fdc66a2d49, user_id=8077ac2b5a214ceab9c7c9d6b1c32012, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:53:26.692843 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d4c66-2d73-4c76-a7e8-2f4ae8348f12 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:53:26.779125 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1217/2452] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:53:26 2022] PUT /identity/v3/projects/b434dc51447f467b8c6521fdc66a2d49/users/8077ac2b5a214ceab9c7c9d6b1c32012/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:53:26.789476 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c78f95-c8c8-42c6-9f12-13fcaca6ab95 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:53:26.789757 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c78f95-c8c8-42c6-9f12-13fcaca6ab95 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:53:26.790031 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c78f95-c8c8-42c6-9f12-13fcaca6ab95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:53:26.826097 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-e6c78f95-c8c8-42c6-9f12-13fcaca6ab95 None None] MFA Rules not processed for user `8077ac2b5a214ceab9c7c9d6b1c32012`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:53:26.827306 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-e6c78f95-c8c8-42c6-9f12-13fcaca6ab95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:53:26.892459 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:53:26.901958 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1236/2453] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:53:26 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:53:26.913837 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d4c66-2d73-4c76-a7e8-2f4ae8348f12 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:53:26.915586 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e85d4c66-2d73-4c76-a7e8-2f4ae8348f12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:53:26.987897 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d4c66-2d73-4c76-a7e8-2f4ae8348f12 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:53:26.991943 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a5c1e6-2845-4fca-9732-150a02156d11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:53:26.992741 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5c1e6-2845-4fca-9732-150a02156d11 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:53:26.993044 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5c1e6-2845-4fca-9732-150a02156d11 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:53:26.993315 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5c1e6-2845-4fca-9732-150a02156d11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:53:27.003266 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a5c1e6-2845-4fca-9732-150a02156d11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:53:27.004999 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a5c1e6-2845-4fca-9732-150a02156d11 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:53:27.008606 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:53:27.014382 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1218/2454] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:53:26 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:53:36.014268 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c78f95-c8c8-42c6-9f12-13fcaca6ab95 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:53:36.076647 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c78f95-c8c8-42c6-9f12-13fcaca6ab95 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:53:36.081731 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58ea58-ba55-4d2c-b250-6096944b5b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:53:36.082360 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58ea58-ba55-4d2c-b250-6096944b5b43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:53:36.082640 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58ea58-ba55-4d2c-b250-6096944b5b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:53:36.082938 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58ea58-ba55-4d2c-b250-6096944b5b43 admin admin] PATH_INFO: `/v3/users/8077ac2b5a214ceab9c7c9d6b1c32012` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:53:36.084065 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d58ea58-ba55-4d2c-b250-6096944b5b43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8077ac2b5a214ceab9c7c9d6b1c32012)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:53:36.085959 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d58ea58-ba55-4d2c-b250-6096944b5b43 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:53:36.284605 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-7d58ea58-ba55-4d2c-b250-6096944b5b43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8077ac2b5a214ceab9c7c9d6b1c32012'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:53:36.292218 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-7d58ea58-ba55-4d2c-b250-6096944b5b43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8077ac2b5a214ceab9c7c9d6b1c32012'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:53:36.305473 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-7d58ea58-ba55-4d2c-b250-6096944b5b43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8077ac2b5a214ceab9c7c9d6b1c32012'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:53:36.306338 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-7d58ea58-ba55-4d2c-b250-6096944b5b43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8077ac2b5a214ceab9c7c9d6b1c32012'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:53:36.367762 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-7d58ea58-ba55-4d2c-b250-6096944b5b43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8077ac2b5a214ceab9c7c9d6b1c32012'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:53:36.374168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1237/2455] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:53:36 2022] DELETE /identity/v3/users/8077ac2b5a214ceab9c7c9d6b1c32012 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:53:36.378798 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e0078-5c5d-454a-b3bb-cc27b37c06cb None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:53:36.379390 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e0078-5c5d-454a-b3bb-cc27b37c06cb None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:53:36.379576 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e0078-5c5d-454a-b3bb-cc27b37c06cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:53:36.399311 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-e12e0078-5c5d-454a-b3bb-cc27b37c06cb None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:53:36.400587 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e12e0078-5c5d-454a-b3bb-cc27b37c06cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:53:36.468541 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1219/2456] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:53:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:53:36.478678 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58ea58-ba55-4d2c-b250-6096944b5b43 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:53:36.480069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-7d58ea58-ba55-4d2c-b250-6096944b5b43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:53:36.536842 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58ea58-ba55-4d2c-b250-6096944b5b43 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:53:36.540368 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2485d0e2-9340-4de6-aa1f-a00b65344263 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:53:36.540963 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485d0e2-9340-4de6-aa1f-a00b65344263 service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:53:36.541238 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485d0e2-9340-4de6-aa1f-a00b65344263 service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:53:36.541511 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485d0e2-9340-4de6-aa1f-a00b65344263 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:53:36.551617 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2485d0e2-9340-4de6-aa1f-a00b65344263 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:53:36.553046 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2485d0e2-9340-4de6-aa1f-a00b65344263 service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:53:36.563912 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1238/2457] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:53:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:53:37.064559 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e0078-5c5d-454a-b3bb-cc27b37c06cb None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:53:37.066511 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e12e0078-5c5d-454a-b3bb-cc27b37c06cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:53:37.082798 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e0078-5c5d-454a-b3bb-cc27b37c06cb None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:53:37.086389 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cda139-d750-4228-a755-c55e69fbd8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:53:37.087150 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cda139-d750-4228-a755-c55e69fbd8df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:53:37.087523 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cda139-d750-4228-a755-c55e69fbd8df admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:53:37.087898 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cda139-d750-4228-a755-c55e69fbd8df admin admin] PATH_INFO: `/v3/projects/b434dc51447f467b8c6521fdc66a2d49` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:53:37.089112 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cda139-d750-4228-a755-c55e69fbd8df admin admin] RBAC: Authorizing `identity:delete_project(project_id=b434dc51447f467b8c6521fdc66a2d49)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:53:37.091122 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cda139-d750-4228-a755-c55e69fbd8df admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:53:37.218409 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e5cda139-d750-4228-a755-c55e69fbd8df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b434dc51447f467b8c6521fdc66a2d49', 'request_id': 'req-e5cda139-d750-4228-a755-c55e69fbd8df'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:53:37.225292 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e5cda139-d750-4228-a755-c55e69fbd8df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b434dc51447f467b8c6521fdc66a2d49', 'request_id': 'req-e5cda139-d750-4228-a755-c55e69fbd8df'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:53:37.338974 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e5cda139-d750-4228-a755-c55e69fbd8df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b434dc51447f467b8c6521fdc66a2d49', 'request_id': 'req-e5cda139-d750-4228-a755-c55e69fbd8df'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:53:37.351876 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e5cda139-d750-4228-a755-c55e69fbd8df admin admin] The token cache is being invalidate because project b434dc51447f467b8c6521fdc66a2d49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:53:37.352162 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e5cda139-d750-4228-a755-c55e69fbd8df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:53:37.352793 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e5cda139-d750-4228-a755-c55e69fbd8df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:53:37.354323 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1220/2458] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:53:37 2022] DELETE /identity/v3/projects/b434dc51447f467b8c6521fdc66a2d49 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:53:46.559498 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=81,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:53:47.348702 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:53:49.515469 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0772f67c-c700-43f1-a8b5-b8f2bfae85c9 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:53:49.516005 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0772f67c-c700-43f1-a8b5-b8f2bfae85c9 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:53:49.516005 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0772f67c-c700-43f1-a8b5-b8f2bfae85c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:53:49.534642 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-0772f67c-c700-43f1-a8b5-b8f2bfae85c9 None None] MFA Rules not processed for user `ad5654cfa65f4cfa9e68832b9cd3d129`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:53:49.536489 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-0772f67c-c700-43f1-a8b5-b8f2bfae85c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:53:49.590696 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1239/2459] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:53:49 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:53:57.874215 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cda139-d750-4228-a755-c55e69fbd8df admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:53:57.875694 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e5cda139-d750-4228-a755-c55e69fbd8df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:53:57.933018 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cda139-d750-4228-a755-c55e69fbd8df admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:53:57.936586 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6759a4f4-eeec-4365-acb6-b05daecbe2f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:53:57.937192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759a4f4-eeec-4365-acb6-b05daecbe2f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:53:57.937473 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759a4f4-eeec-4365-acb6-b05daecbe2f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:53:57.937747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759a4f4-eeec-4365-acb6-b05daecbe2f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:53:57.939465 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-6759a4f4-eeec-4365-acb6-b05daecbe2f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:53:58.003166 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6759a4f4-eeec-4365-acb6-b05daecbe2f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:53:58.005204 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6759a4f4-eeec-4365-acb6-b05daecbe2f7 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:53:58.014680 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1221/2460] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:53:57 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:53:59.583548 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=25 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:54:08.010409 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=50 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:54:25.537771 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9138e1d-0c3f-42c2-ad23-3d61133db211 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:54:25.538293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9138e1d-0c3f-42c2-ad23-3d61133db211 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:54:25.538479 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9138e1d-0c3f-42c2-ad23-3d61133db211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:54:25.560520 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-a9138e1d-0c3f-42c2-ad23-3d61133db211 None None] MFA Rules not processed for user `676b2a6bbf6e4df1b47dba2a9b8e8862`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:54:25.561875 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-a9138e1d-0c3f-42c2-ad23-3d61133db211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:54:25.627882 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1240/2461] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:54:25 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:54:26.364913 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6759a4f4-eeec-4365-acb6-b05daecbe2f7 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:54:26.372370 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-6759a4f4-eeec-4365-acb6-b05daecbe2f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:54:26.379124 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6759a4f4-eeec-4365-acb6-b05daecbe2f7 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:54:26.382964 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab6a36-b2f6-4160-af46-32884d9e07be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:54:26.384208 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab6a36-b2f6-4160-af46-32884d9e07be service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:54:26.384580 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab6a36-b2f6-4160-af46-32884d9e07be service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:54:26.384780 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab6a36-b2f6-4160-af46-32884d9e07be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:54:26.386502 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-0cab6a36-b2f6-4160-af46-32884d9e07be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:54:26.467356 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab6a36-b2f6-4160-af46-32884d9e07be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:54:26.468911 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab6a36-b2f6-4160-af46-32884d9e07be service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:54:26.483106 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1222/2462] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:54:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:54:27.622809 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9138e1d-0c3f-42c2-ad23-3d61133db211 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:54:27.632313 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9138e1d-0c3f-42c2-ad23-3d61133db211 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:54:27.636247 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e4363-dd0a-4fee-8314-bdc817d45560 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:54:27.637082 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e4363-dd0a-4fee-8314-bdc817d45560 service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:54:27.637369 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e4363-dd0a-4fee-8314-bdc817d45560 service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:54:27.637639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e4363-dd0a-4fee-8314-bdc817d45560 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:54:27.639446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-d43e4363-dd0a-4fee-8314-bdc817d45560 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:54:27.653550 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43e4363-dd0a-4fee-8314-bdc817d45560 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:54:27.654965 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43e4363-dd0a-4fee-8314-bdc817d45560 service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:54:27.666788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1241/2463] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:54:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:54:32.152141 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2a615-a1a1-4f48-adc1-8efa7c2f79d9 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:54:32.152677 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2a615-a1a1-4f48-adc1-8efa7c2f79d9 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:54:32.152677 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2a615-a1a1-4f48-adc1-8efa7c2f79d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:54:32.172167 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-5af2a615-a1a1-4f48-adc1-8efa7c2f79d9 None None] MFA Rules not processed for user `bd24d209dc9644e8b2860a0e676acd3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:54:32.173318 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-5af2a615-a1a1-4f48-adc1-8efa7c2f79d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:54:32.229426 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1223/2464] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:54:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:54:36.996688 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e4363-dd0a-4fee-8314-bdc817d45560 service neutron] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:54:36.998136 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-d43e4363-dd0a-4fee-8314-bdc817d45560 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:54:37.054714 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e4363-dd0a-4fee-8314-bdc817d45560 service neutron] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:54:37.058424 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0cac57-f0e4-41bc-a001-7ab22b0f4ab9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:54:37.059033 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0cac57-f0e4-41bc-a001-7ab22b0f4ab9 service cinder] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:54:37.059343 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0cac57-f0e4-41bc-a001-7ab22b0f4ab9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:54:37.059614 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0cac57-f0e4-41bc-a001-7ab22b0f4ab9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:54:37.061318 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-3b0cac57-f0e4-41bc-a001-7ab22b0f4ab9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:54:37.072443 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0cac57-f0e4-41bc-a001-7ab22b0f4ab9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:54:37.082316 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0cac57-f0e4-41bc-a001-7ab22b0f4ab9 service cinder] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:54:37.087537 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1242/2465] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:54:36 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:54:42.221860 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=59 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:54:47.078602 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=57 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:55:01.419477 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af2a615-a1a1-4f48-adc1-8efa7c2f79d9 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:01.420904 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-5af2a615-a1a1-4f48-adc1-8efa7c2f79d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:01.436777 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af2a615-a1a1-4f48-adc1-8efa7c2f79d9 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:01.442140 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b78e14-d273-4e99-a33e-a9a5523e4ccc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:01.442742 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b78e14-d273-4e99-a33e-a9a5523e4ccc service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:01.443017 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b78e14-d273-4e99-a33e-a9a5523e4ccc service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:01.443366 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b78e14-d273-4e99-a33e-a9a5523e4ccc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:01.445844 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-64b78e14-d273-4e99-a33e-a9a5523e4ccc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:01.457916 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b78e14-d273-4e99-a33e-a9a5523e4ccc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:01.460191 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b78e14-d273-4e99-a33e-a9a5523e4ccc service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:01.470507 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1224/2466] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:55:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:55:03.108304 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0cac57-f0e4-41bc-a001-7ab22b0f4ab9 service cinder] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:03.118087 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0cac57-f0e4-41bc-a001-7ab22b0f4ab9 service cinder] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:03.121858 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a7d81-356f-441f-b59f-3c58adb982d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:03.122465 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a7d81-356f-441f-b59f-3c58adb982d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:03.122810 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a7d81-356f-441f-b59f-3c58adb982d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:03.123090 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a7d81-356f-441f-b59f-3c58adb982d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:03.124965 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-158a7d81-356f-441f-b59f-3c58adb982d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:03.187679 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158a7d81-356f-441f-b59f-3c58adb982d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:03.189094 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158a7d81-356f-441f-b59f-3c58adb982d3 service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:03.201980 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1243/2467] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:55:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:55:06.891286 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2d448-ae25-49e7-b7b6-5966198202e7 None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:06.891793 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2d448-ae25-49e7-b7b6-5966198202e7 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:06.891939 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2d448-ae25-49e7-b7b6-5966198202e7 None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:06.893366 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1225/2468] 10.222.0.33 () {54 vars in 921 bytes} [Mon Oct 3 12:55:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:55:06.904405 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73297af-7992-42d2-9073-f77f1c55d720 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:06.907445 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73297af-7992-42d2-9073-f77f1c55d720 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:06.907445 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73297af-7992-42d2-9073-f77f1c55d720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:06.925970 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-a73297af-7992-42d2-9073-f77f1c55d720 None None] MFA Rules not processed for user `25eef64e92e9422ea62508c3eb0a197e`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:55:06.927370 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-a73297af-7992-42d2-9073-f77f1c55d720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:06.994669 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1244/2469] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Oct 3 12:55:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:55:07.005745 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc2d448-ae25-49e7-b7b6-5966198202e7 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:07.007869 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-8cc2d448-ae25-49e7-b7b6-5966198202e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:07.024828 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc2d448-ae25-49e7-b7b6-5966198202e7 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:07.029874 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97a7a05-4f4f-46a4-85d2-c3d02af95040 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25eef64e92e9422ea62508c3eb0a197e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:07.030468 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a7a05-4f4f-46a4-85d2-c3d02af95040 service placement] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:07.030747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a7a05-4f4f-46a4-85d2-c3d02af95040 service placement] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:07.031020 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a7a05-4f4f-46a4-85d2-c3d02af95040 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:07.040382 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97a7a05-4f4f-46a4-85d2-c3d02af95040 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:07.043407 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97a7a05-4f4f-46a4-85d2-c3d02af95040 service placement] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:07.055272 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73297af-7992-42d2-9073-f77f1c55d720 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:07.056871 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-a73297af-7992-42d2-9073-f77f1c55d720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:07.060159 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1226/2470] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:55:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:55:07.071551 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73297af-7992-42d2-9073-f77f1c55d720 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:07.076365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceda5d0-3b4f-4283-a19b-9fcf230fed55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:07.077156 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceda5d0-3b4f-4283-a19b-9fcf230fed55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:07.077302 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceda5d0-3b4f-4283-a19b-9fcf230fed55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:07.077579 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceda5d0-3b4f-4283-a19b-9fcf230fed55 admin admin] PATH_INFO: `/v3/users/ad5654cfa65f4cfa9e68832b9cd3d129` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:07.078804 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceda5d0-3b4f-4283-a19b-9fcf230fed55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad5654cfa65f4cfa9e68832b9cd3d129)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:07.081075 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceda5d0-3b4f-4283-a19b-9fcf230fed55 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:07.288095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-bceda5d0-3b4f-4283-a19b-9fcf230fed55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad5654cfa65f4cfa9e68832b9cd3d129'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:07.294992 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-bceda5d0-3b4f-4283-a19b-9fcf230fed55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad5654cfa65f4cfa9e68832b9cd3d129'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:07.306795 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-bceda5d0-3b4f-4283-a19b-9fcf230fed55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad5654cfa65f4cfa9e68832b9cd3d129'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:07.307514 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-bceda5d0-3b4f-4283-a19b-9fcf230fed55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad5654cfa65f4cfa9e68832b9cd3d129'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:07.373037 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-bceda5d0-3b4f-4283-a19b-9fcf230fed55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad5654cfa65f4cfa9e68832b9cd3d129'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:07.379258 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1245/2471] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:55:07 2022] DELETE /identity/v3/users/ad5654cfa65f4cfa9e68832b9cd3d129 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:55:07.629157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97a7a05-4f4f-46a4-85d2-c3d02af95040 service placement] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:07.640471 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97a7a05-4f4f-46a4-85d2-c3d02af95040 service placement] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:07.698468 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a18c4-ff7b-46e8-a6dc-dc5cc6455dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:07.699102 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a18c4-ff7b-46e8-a6dc-dc5cc6455dbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:07.699409 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a18c4-ff7b-46e8-a6dc-dc5cc6455dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:07.699683 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a18c4-ff7b-46e8-a6dc-dc5cc6455dbc admin admin] PATH_INFO: `/v3/projects/3f2b18d217d14d39886718cf209eb576` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:07.700780 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a18c4-ff7b-46e8-a6dc-dc5cc6455dbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f2b18d217d14d39886718cf209eb576)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:07.702664 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a18c4-ff7b-46e8-a6dc-dc5cc6455dbc admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:07.801007 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4f4a18c4-ff7b-46e8-a6dc-dc5cc6455dbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f2b18d217d14d39886718cf209eb576', 'request_id': 'req-4f4a18c4-ff7b-46e8-a6dc-dc5cc6455dbc'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:07.808012 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4f4a18c4-ff7b-46e8-a6dc-dc5cc6455dbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f2b18d217d14d39886718cf209eb576', 'request_id': 'req-4f4a18c4-ff7b-46e8-a6dc-dc5cc6455dbc'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:07.890885 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4f4a18c4-ff7b-46e8-a6dc-dc5cc6455dbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f2b18d217d14d39886718cf209eb576', 'request_id': 'req-4f4a18c4-ff7b-46e8-a6dc-dc5cc6455dbc'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:07.903114 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4f4a18c4-ff7b-46e8-a6dc-dc5cc6455dbc admin admin] The token cache is being invalidate because project 3f2b18d217d14d39886718cf209eb576 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:55:07.903714 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4f4a18c4-ff7b-46e8-a6dc-dc5cc6455dbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:07.904237 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-4f4a18c4-ff7b-46e8-a6dc-dc5cc6455dbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:07.905738 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1227/2472] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:55:07 2022] DELETE /identity/v3/projects/3f2b18d217d14d39886718cf209eb576 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:55:07.917598 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d0e67a-31ab-4d79-bcd4-5a2e5bf27b69 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:07.918139 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d0e67a-31ab-4d79-bcd4-5a2e5bf27b69 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:07.918139 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d0e67a-31ab-4d79-bcd4-5a2e5bf27b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:07.926119 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:55:07.941657 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-84d0e67a-31ab-4d79-bcd4-5a2e5bf27b69 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:55:07.942932 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-84d0e67a-31ab-4d79-bcd4-5a2e5bf27b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:08.009600 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1246/2473] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:55:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:55:08.024098 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c5bb5-8100-43e3-adc6-4b4ad60f5fb3 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:08.024409 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c5bb5-8100-43e3-adc6-4b4ad60f5fb3 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:08.024691 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c5bb5-8100-43e3-adc6-4b4ad60f5fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:08.044309 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-513c5bb5-8100-43e3-adc6-4b4ad60f5fb3 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:55:08.045514 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-513c5bb5-8100-43e3-adc6-4b4ad60f5fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:08.054065 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1228/2474] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:55:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:55:08.058331 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d0e67a-31ab-4d79-bcd4-5a2e5bf27b69 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:08.067988 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d0e67a-31ab-4d79-bcd4-5a2e5bf27b69 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:08.073219 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef0fb9e-13f8-4b74-96f2-69cdb8d7934d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:08.073817 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0fb9e-13f8-4b74-96f2-69cdb8d7934d admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:08.074090 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0fb9e-13f8-4b74-96f2-69cdb8d7934d admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:08.074359 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0fb9e-13f8-4b74-96f2-69cdb8d7934d admin admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:08.074993 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef0fb9e-13f8-4b74-96f2-69cdb8d7934d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:55:08.075294 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef0fb9e-13f8-4b74-96f2-69cdb8d7934d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:08.077024 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef0fb9e-13f8-4b74-96f2-69cdb8d7934d admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:08.088588 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1247/2475] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:55:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:55:08.092799 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513c5bb5-8100-43e3-adc6-4b4ad60f5fb3 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:08.102657 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513c5bb5-8100-43e3-adc6-4b4ad60f5fb3 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:08.107515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b6390-816f-459b-9a10-b6cd2cdc817e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:08.113013 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b6390-816f-459b-9a10-b6cd2cdc817e admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:08.113013 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b6390-816f-459b-9a10-b6cd2cdc817e admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:08.113013 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b6390-816f-459b-9a10-b6cd2cdc817e admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:08.113013 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07b6390-816f-459b-9a10-b6cd2cdc817e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:08.113013 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07b6390-816f-459b-9a10-b6cd2cdc817e admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:08.216330 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1229/2476] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:55:08 2022] POST /identity/v3/projects => generated 367 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:55:08.220226 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef0fb9e-13f8-4b74-96f2-69cdb8d7934d admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:08.230532 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef0fb9e-13f8-4b74-96f2-69cdb8d7934d admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:08.286005 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3425a0-ccfa-40b7-8e03-7840c4f08749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:08.286616 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3425a0-ccfa-40b7-8e03-7840c4f08749 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:08.286896 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3425a0-ccfa-40b7-8e03-7840c4f08749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:08.287696 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3425a0-ccfa-40b7-8e03-7840c4f08749 admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:08.288399 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3425a0-ccfa-40b7-8e03-7840c4f08749 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:08.290309 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3425a0-ccfa-40b7-8e03-7840c4f08749 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:08.374448 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1248/2477] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:55:08 2022] POST /identity/v3/users => generated 284 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:55:08.378607 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b6390-816f-459b-9a10-b6cd2cdc817e admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:08.389259 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b6390-816f-459b-9a10-b6cd2cdc817e admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:08.393902 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c4d41-b9d0-44df-ae1c-fd36923fc0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:08.394534 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c4d41-b9d0-44df-ae1c-fd36923fc0f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:08.394804 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c4d41-b9d0-44df-ae1c-fd36923fc0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:08.395075 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c4d41-b9d0-44df-ae1c-fd36923fc0f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:08.395750 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c4d41-b9d0-44df-ae1c-fd36923fc0f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:08.397540 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c4d41-b9d0-44df-ae1c-fd36923fc0f6 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:08.407301 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1230/2478] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:55:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:55:08.417018 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3425a0-ccfa-40b7-8e03-7840c4f08749 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:08.427053 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3425a0-ccfa-40b7-8e03-7840c4f08749 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:08.432759 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823712e0-9bfa-4745-a2a8-0c59091e73a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:08.433360 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823712e0-9bfa-4745-a2a8-0c59091e73a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:08.433640 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823712e0-9bfa-4745-a2a8-0c59091e73a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:08.433914 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823712e0-9bfa-4745-a2a8-0c59091e73a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:08.434554 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823712e0-9bfa-4745-a2a8-0c59091e73a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:08.436181 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823712e0-9bfa-4745-a2a8-0c59091e73a0 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:08.447727 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1249/2479] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:55:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:55:08.451939 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c4d41-b9d0-44df-ae1c-fd36923fc0f6 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:08.462489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c4d41-b9d0-44df-ae1c-fd36923fc0f6 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:08.467075 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb10fbe3-4480-4956-9123-6ddefb01bcf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:08.467703 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10fbe3-4480-4956-9123-6ddefb01bcf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:08.467981 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10fbe3-4480-4956-9123-6ddefb01bcf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:08.468256 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10fbe3-4480-4956-9123-6ddefb01bcf2 admin admin] PATH_INFO: `/v3/projects/db2ee49bc8a44e8f8aaa780dd4f465c0/users/039ad1a45a3d4b96a5acf229db3315de/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:08.483948 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb10fbe3-4480-4956-9123-6ddefb01bcf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db2ee49bc8a44e8f8aaa780dd4f465c0, user_id=039ad1a45a3d4b96a5acf229db3315de, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:08.486565 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb10fbe3-4480-4956-9123-6ddefb01bcf2 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:08.549717 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:55:08.554344 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1231/2480] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:55:08 2022] PUT /identity/v3/projects/db2ee49bc8a44e8f8aaa780dd4f465c0/users/039ad1a45a3d4b96a5acf229db3315de/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:55:08.566120 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e982795-574f-4aec-b4d0-a5cd79536371 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:08.566409 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e982795-574f-4aec-b4d0-a5cd79536371 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:08.566682 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e982795-574f-4aec-b4d0-a5cd79536371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:08.595545 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-9e982795-574f-4aec-b4d0-a5cd79536371 None None] MFA Rules not processed for user `039ad1a45a3d4b96a5acf229db3315de`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:55:08.596832 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-9e982795-574f-4aec-b4d0-a5cd79536371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:08.654858 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:55:08.659887 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1250/2481] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:55:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:55:08.665085 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb10fbe3-4480-4956-9123-6ddefb01bcf2 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:08.726907 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb10fbe3-4480-4956-9123-6ddefb01bcf2 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:08.731489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc13c857-0a84-485f-b792-51bdbd5cfd95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:08.732098 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc13c857-0a84-485f-b792-51bdbd5cfd95 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:08.732375 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc13c857-0a84-485f-b792-51bdbd5cfd95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:08.732645 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc13c857-0a84-485f-b792-51bdbd5cfd95 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:08.733274 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc13c857-0a84-485f-b792-51bdbd5cfd95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:08.735144 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc13c857-0a84-485f-b792-51bdbd5cfd95 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:08.831200 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1232/2482] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:55:08 2022] POST /identity/v3/projects => generated 367 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:55:08.836116 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e982795-574f-4aec-b4d0-a5cd79536371 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:08.847133 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e982795-574f-4aec-b4d0-a5cd79536371 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:08.907595 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f713a7dd-82a7-4c46-a51f-4d5d065c4f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:08.908208 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f713a7dd-82a7-4c46-a51f-4d5d065c4f97 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:08.908494 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f713a7dd-82a7-4c46-a51f-4d5d065c4f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:08.908765 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f713a7dd-82a7-4c46-a51f-4d5d065c4f97 admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:08.909416 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f713a7dd-82a7-4c46-a51f-4d5d065c4f97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:08.911341 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f713a7dd-82a7-4c46-a51f-4d5d065c4f97 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:08.993566 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1251/2483] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:55:08 2022] POST /identity/v3/users => generated 290 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:55:08.997540 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc13c857-0a84-485f-b792-51bdbd5cfd95 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:09.008315 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc13c857-0a84-485f-b792-51bdbd5cfd95 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:09.013036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12bf40-d411-4893-b218-5748ff91dc30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:09.015442 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12bf40-d411-4893-b218-5748ff91dc30 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:09.015442 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12bf40-d411-4893-b218-5748ff91dc30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:09.015442 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12bf40-d411-4893-b218-5748ff91dc30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:09.015442 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12bf40-d411-4893-b218-5748ff91dc30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:09.016425 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12bf40-d411-4893-b218-5748ff91dc30 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:09.027534 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1233/2484] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:55:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:55:09.031745 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f713a7dd-82a7-4c46-a51f-4d5d065c4f97 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:09.041687 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f713a7dd-82a7-4c46-a51f-4d5d065c4f97 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:09.046266 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf11da-8af4-4bd1-850a-32eea7a494de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:09.046907 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf11da-8af4-4bd1-850a-32eea7a494de admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:09.047185 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf11da-8af4-4bd1-850a-32eea7a494de admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:09.047519 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf11da-8af4-4bd1-850a-32eea7a494de admin admin] PATH_INFO: `/v3/projects/851dbdd1c16f4e31b40374cc3ef9b953/users/021f9610f87e45eaac59c0e240d7c90a/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:09.062137 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdf11da-8af4-4bd1-850a-32eea7a494de admin admin] RBAC: Authorizing `identity:create_grant(project_id=851dbdd1c16f4e31b40374cc3ef9b953, user_id=021f9610f87e45eaac59c0e240d7c90a, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:09.064807 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdf11da-8af4-4bd1-850a-32eea7a494de admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:09.124820 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1252/2485] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:55:09 2022] PUT /identity/v3/projects/851dbdd1c16f4e31b40374cc3ef9b953/users/021f9610f87e45eaac59c0e240d7c90a/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:55:09.128819 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12bf40-d411-4893-b218-5748ff91dc30 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:09.139736 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12bf40-d411-4893-b218-5748ff91dc30 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:09.194950 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed55be2-5b5b-45ed-90fc-a5a724c9f1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:09.195585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed55be2-5b5b-45ed-90fc-a5a724c9f1af admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:09.195863 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed55be2-5b5b-45ed-90fc-a5a724c9f1af admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:09.196134 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed55be2-5b5b-45ed-90fc-a5a724c9f1af admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:09.196791 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed55be2-5b5b-45ed-90fc-a5a724c9f1af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:09.199019 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed55be2-5b5b-45ed-90fc-a5a724c9f1af admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:09.213706 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1234/2486] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:55:09 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:55:09.217903 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf11da-8af4-4bd1-850a-32eea7a494de admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:09.229647 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf11da-8af4-4bd1-850a-32eea7a494de admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:09.234435 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c309756c-a0d1-4e10-8229-4b634f12d4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:09.235089 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c309756c-a0d1-4e10-8229-4b634f12d4aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:09.235398 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c309756c-a0d1-4e10-8229-4b634f12d4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:09.235703 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c309756c-a0d1-4e10-8229-4b634f12d4aa admin admin] PATH_INFO: `/v3/projects/851dbdd1c16f4e31b40374cc3ef9b953/users/021f9610f87e45eaac59c0e240d7c90a/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:09.237620 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c309756c-a0d1-4e10-8229-4b634f12d4aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=851dbdd1c16f4e31b40374cc3ef9b953, user_id=021f9610f87e45eaac59c0e240d7c90a, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:09.240857 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c309756c-a0d1-4e10-8229-4b634f12d4aa admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:09.251568 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1253/2487] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:55:09 2022] PUT /identity/v3/projects/851dbdd1c16f4e31b40374cc3ef9b953/users/021f9610f87e45eaac59c0e240d7c90a/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:55:09.262348 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b66f90-6c2e-4a86-9525-5c907607232a None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:09.262631 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b66f90-6c2e-4a86-9525-5c907607232a None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:09.262904 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b66f90-6c2e-4a86-9525-5c907607232a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:09.291228 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-e2b66f90-6c2e-4a86-9525-5c907607232a None None] MFA Rules not processed for user `021f9610f87e45eaac59c0e240d7c90a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:55:09.299666 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e2b66f90-6c2e-4a86-9525-5c907607232a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:09.351449 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:55:09.360126 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1235/2488] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:55:09 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:55:09.377162 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c309756c-a0d1-4e10-8229-4b634f12d4aa admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:09.379057 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-c309756c-a0d1-4e10-8229-4b634f12d4aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:09.436076 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c309756c-a0d1-4e10-8229-4b634f12d4aa admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:09.439776 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7b5829-df03-4b9e-b425-eb7a6ccaea5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:09.440383 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b5829-df03-4b9e-b425-eb7a6ccaea5f service cinder] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:09.440662 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b5829-df03-4b9e-b425-eb7a6ccaea5f service cinder] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:09.440935 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b5829-df03-4b9e-b425-eb7a6ccaea5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:09.453595 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7b5829-df03-4b9e-b425-eb7a6ccaea5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:09.462121 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7b5829-df03-4b9e-b425-eb7a6ccaea5f service cinder] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:09.465941 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1254/2489] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:55:09 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:55:12.327543 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b66f90-6c2e-4a86-9525-5c907607232a None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:12.391294 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b66f90-6c2e-4a86-9525-5c907607232a None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:12.395728 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac17003-01da-4cb5-8974-d7f6eea4ea90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:12.396358 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac17003-01da-4cb5-8974-d7f6eea4ea90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:12.396764 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac17003-01da-4cb5-8974-d7f6eea4ea90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:12.397038 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac17003-01da-4cb5-8974-d7f6eea4ea90 admin admin] PATH_INFO: `/v3/users/039ad1a45a3d4b96a5acf229db3315de` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:12.398177 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac17003-01da-4cb5-8974-d7f6eea4ea90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=039ad1a45a3d4b96a5acf229db3315de)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:12.400118 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac17003-01da-4cb5-8974-d7f6eea4ea90 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:12.584404 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9ac17003-01da-4cb5-8974-d7f6eea4ea90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '039ad1a45a3d4b96a5acf229db3315de'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:12.596259 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9ac17003-01da-4cb5-8974-d7f6eea4ea90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '039ad1a45a3d4b96a5acf229db3315de'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:12.596914 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9ac17003-01da-4cb5-8974-d7f6eea4ea90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '039ad1a45a3d4b96a5acf229db3315de'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:12.604305 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9ac17003-01da-4cb5-8974-d7f6eea4ea90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '039ad1a45a3d4b96a5acf229db3315de'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:12.664477 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9ac17003-01da-4cb5-8974-d7f6eea4ea90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '039ad1a45a3d4b96a5acf229db3315de'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:12.672174 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1236/2490] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:55:12 2022] DELETE /identity/v3/users/039ad1a45a3d4b96a5acf229db3315de => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:55:12.675273 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c402f-0c50-46ca-a267-382a7ef5831b None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:12.675705 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c402f-0c50-46ca-a267-382a7ef5831b None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:12.675859 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c402f-0c50-46ca-a267-382a7ef5831b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:12.695245 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-af9c402f-0c50-46ca-a267-382a7ef5831b None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:55:12.698176 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-af9c402f-0c50-46ca-a267-382a7ef5831b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:12.744943 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:55:12.757006 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1255/2491] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:55:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:55:12.766846 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac17003-01da-4cb5-8974-d7f6eea4ea90 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:12.771201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-9ac17003-01da-4cb5-8974-d7f6eea4ea90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:12.828888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac17003-01da-4cb5-8974-d7f6eea4ea90 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:12.832457 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38443b78-35f3-402d-875c-ce4a4531924c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:12.833058 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38443b78-35f3-402d-875c-ce4a4531924c service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:12.833332 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38443b78-35f3-402d-875c-ce4a4531924c service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:12.833600 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38443b78-35f3-402d-875c-ce4a4531924c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:12.842535 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38443b78-35f3-402d-875c-ce4a4531924c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:12.843986 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38443b78-35f3-402d-875c-ce4a4531924c service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:12.853963 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1237/2492] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:55:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:55:13.232663 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9c402f-0c50-46ca-a267-382a7ef5831b None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:13.234105 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-af9c402f-0c50-46ca-a267-382a7ef5831b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:13.247539 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9c402f-0c50-46ca-a267-382a7ef5831b None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:13.252743 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf04dec0-55e5-4eb1-9194-e47c2cf63c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:13.253406 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04dec0-55e5-4eb1-9194-e47c2cf63c81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:13.253692 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04dec0-55e5-4eb1-9194-e47c2cf63c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:13.253987 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04dec0-55e5-4eb1-9194-e47c2cf63c81 admin admin] PATH_INFO: `/v3/projects/db2ee49bc8a44e8f8aaa780dd4f465c0` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:13.255172 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf04dec0-55e5-4eb1-9194-e47c2cf63c81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db2ee49bc8a44e8f8aaa780dd4f465c0)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:13.257162 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf04dec0-55e5-4eb1-9194-e47c2cf63c81 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:13.391087 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-bf04dec0-55e5-4eb1-9194-e47c2cf63c81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db2ee49bc8a44e8f8aaa780dd4f465c0', 'request_id': 'req-bf04dec0-55e5-4eb1-9194-e47c2cf63c81'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:13.398157 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-bf04dec0-55e5-4eb1-9194-e47c2cf63c81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db2ee49bc8a44e8f8aaa780dd4f465c0', 'request_id': 'req-bf04dec0-55e5-4eb1-9194-e47c2cf63c81'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:13.467608 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-bf04dec0-55e5-4eb1-9194-e47c2cf63c81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db2ee49bc8a44e8f8aaa780dd4f465c0', 'request_id': 'req-bf04dec0-55e5-4eb1-9194-e47c2cf63c81'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:13.478214 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-bf04dec0-55e5-4eb1-9194-e47c2cf63c81 admin admin] The token cache is being invalidate because project db2ee49bc8a44e8f8aaa780dd4f465c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:55:13.478532 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-bf04dec0-55e5-4eb1-9194-e47c2cf63c81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:13.479155 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-bf04dec0-55e5-4eb1-9194-e47c2cf63c81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:13.480764 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1256/2493] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:55:13 2022] DELETE /identity/v3/projects/db2ee49bc8a44e8f8aaa780dd4f465c0 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:55:13.484826 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38443b78-35f3-402d-875c-ce4a4531924c service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:13.496227 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38443b78-35f3-402d-875c-ce4a4531924c service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:13.509190 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:55:13.554935 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a4037-dfa0-45b2-8c76-5f7b75111c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:13.555608 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a4037-dfa0-45b2-8c76-5f7b75111c5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:13.555887 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a4037-dfa0-45b2-8c76-5f7b75111c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:13.556158 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a4037-dfa0-45b2-8c76-5f7b75111c5a admin admin] PATH_INFO: `/v3/users/021f9610f87e45eaac59c0e240d7c90a` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:13.557278 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998a4037-dfa0-45b2-8c76-5f7b75111c5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=021f9610f87e45eaac59c0e240d7c90a)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:13.559192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998a4037-dfa0-45b2-8c76-5f7b75111c5a admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:13.751199 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-998a4037-dfa0-45b2-8c76-5f7b75111c5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '021f9610f87e45eaac59c0e240d7c90a'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:13.763672 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-998a4037-dfa0-45b2-8c76-5f7b75111c5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '021f9610f87e45eaac59c0e240d7c90a'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:13.764336 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-998a4037-dfa0-45b2-8c76-5f7b75111c5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '021f9610f87e45eaac59c0e240d7c90a'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:13.771257 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-998a4037-dfa0-45b2-8c76-5f7b75111c5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '021f9610f87e45eaac59c0e240d7c90a'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:13.850547 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-998a4037-dfa0-45b2-8c76-5f7b75111c5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '021f9610f87e45eaac59c0e240d7c90a'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:13.856876 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1238/2494] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:55:13 2022] DELETE /identity/v3/users/021f9610f87e45eaac59c0e240d7c90a => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:55:14.246759 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf04dec0-55e5-4eb1-9194-e47c2cf63c81 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:14.248516 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-bf04dec0-55e5-4eb1-9194-e47c2cf63c81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:14.266023 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf04dec0-55e5-4eb1-9194-e47c2cf63c81 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:14.323298 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f55f9c-2b38-4afb-b197-221e7d7a8819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:14.323981 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f55f9c-2b38-4afb-b197-221e7d7a8819 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:14.324258 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f55f9c-2b38-4afb-b197-221e7d7a8819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:14.324533 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f55f9c-2b38-4afb-b197-221e7d7a8819 admin admin] PATH_INFO: `/v3/projects/851dbdd1c16f4e31b40374cc3ef9b953` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:14.325628 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f55f9c-2b38-4afb-b197-221e7d7a8819 admin admin] RBAC: Authorizing `identity:delete_project(project_id=851dbdd1c16f4e31b40374cc3ef9b953)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:14.327547 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f55f9c-2b38-4afb-b197-221e7d7a8819 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:14.421867 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-d7f55f9c-2b38-4afb-b197-221e7d7a8819 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '851dbdd1c16f4e31b40374cc3ef9b953', 'request_id': 'req-d7f55f9c-2b38-4afb-b197-221e7d7a8819'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:14.429075 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-d7f55f9c-2b38-4afb-b197-221e7d7a8819 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '851dbdd1c16f4e31b40374cc3ef9b953', 'request_id': 'req-d7f55f9c-2b38-4afb-b197-221e7d7a8819'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:14.501554 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-d7f55f9c-2b38-4afb-b197-221e7d7a8819 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '851dbdd1c16f4e31b40374cc3ef9b953', 'request_id': 'req-d7f55f9c-2b38-4afb-b197-221e7d7a8819'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:14.512585 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-d7f55f9c-2b38-4afb-b197-221e7d7a8819 admin admin] The token cache is being invalidate because project 851dbdd1c16f4e31b40374cc3ef9b953 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:55:14.512822 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-d7f55f9c-2b38-4afb-b197-221e7d7a8819 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:14.513450 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-d7f55f9c-2b38-4afb-b197-221e7d7a8819 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:14.514938 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1257/2495] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:55:14 2022] DELETE /identity/v3/projects/851dbdd1c16f4e31b40374cc3ef9b953 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:55:14.519943 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e31c8e-a5f0-4c4b-ad4f-db33eeb913c5 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:14.520407 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e31c8e-a5f0-4c4b-ad4f-db33eeb913c5 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:14.520509 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e31c8e-a5f0-4c4b-ad4f-db33eeb913c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:14.543064 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-85e31c8e-a5f0-4c4b-ad4f-db33eeb913c5 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:55:14.544325 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-85e31c8e-a5f0-4c4b-ad4f-db33eeb913c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:14.606019 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1239/2496] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:55:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:55:14.613925 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5bc661-4f3a-43a8-97c5-02af9ca71681 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:14.614214 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5bc661-4f3a-43a8-97c5-02af9ca71681 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:14.614492 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5bc661-4f3a-43a8-97c5-02af9ca71681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:14.629149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:55:14.634551 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-da5bc661-4f3a-43a8-97c5-02af9ca71681 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:55:14.635793 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-da5bc661-4f3a-43a8-97c5-02af9ca71681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:14.643677 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1258/2497] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:55:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:55:14.648675 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e31c8e-a5f0-4c4b-ad4f-db33eeb913c5 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:14.658519 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e31c8e-a5f0-4c4b-ad4f-db33eeb913c5 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:14.663799 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb222d91-5f8c-4aaa-a8a8-524310bdd193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:14.668627 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb222d91-5f8c-4aaa-a8a8-524310bdd193 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:14.668627 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb222d91-5f8c-4aaa-a8a8-524310bdd193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:14.668627 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb222d91-5f8c-4aaa-a8a8-524310bdd193 admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:14.668627 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb222d91-5f8c-4aaa-a8a8-524310bdd193 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:55:14.668627 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb222d91-5f8c-4aaa-a8a8-524310bdd193 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:14.668627 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb222d91-5f8c-4aaa-a8a8-524310bdd193 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:14.678698 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1240/2498] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:55:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:55:14.682905 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5bc661-4f3a-43a8-97c5-02af9ca71681 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:14.692767 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5bc661-4f3a-43a8-97c5-02af9ca71681 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:14.697517 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7290c-7edd-41ed-89cd-c1366efa0a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:14.698208 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7290c-7edd-41ed-89cd-c1366efa0a13 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:14.698495 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7290c-7edd-41ed-89cd-c1366efa0a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:14.698770 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7290c-7edd-41ed-89cd-c1366efa0a13 admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:14.699444 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a7290c-7edd-41ed-89cd-c1366efa0a13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:14.701356 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a7290c-7edd-41ed-89cd-c1366efa0a13 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:14.780122 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1259/2499] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:55:14 2022] POST /identity/v3/projects => generated 365 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:55:14.784315 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb222d91-5f8c-4aaa-a8a8-524310bdd193 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:14.794895 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb222d91-5f8c-4aaa-a8a8-524310bdd193 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:14.814949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:55:14.851117 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e193296f-5c32-44d9-8646-c0ec6e963b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:14.851753 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e193296f-5c32-44d9-8646-c0ec6e963b31 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:14.852195 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e193296f-5c32-44d9-8646-c0ec6e963b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:14.852474 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e193296f-5c32-44d9-8646-c0ec6e963b31 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:14.853104 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e193296f-5c32-44d9-8646-c0ec6e963b31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:14.855024 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e193296f-5c32-44d9-8646-c0ec6e963b31 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:14.959998 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1241/2500] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:55:14 2022] POST /identity/v3/users => generated 283 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:55:14.964102 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7290c-7edd-41ed-89cd-c1366efa0a13 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:14.974333 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7290c-7edd-41ed-89cd-c1366efa0a13 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:14.979479 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a3a866-e82a-4741-9c90-46825073adad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:14.980559 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3a866-e82a-4741-9c90-46825073adad admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:14.980559 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3a866-e82a-4741-9c90-46825073adad admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:14.980744 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3a866-e82a-4741-9c90-46825073adad admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:14.981223 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a3a866-e82a-4741-9c90-46825073adad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:14.982807 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a3a866-e82a-4741-9c90-46825073adad admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:14.992881 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1260/2501] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:55:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:55:14.996811 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e193296f-5c32-44d9-8646-c0ec6e963b31 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:15.007258 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e193296f-5c32-44d9-8646-c0ec6e963b31 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:15.011808 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2958745b-0cec-4050-8e51-420e4c11d225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:15.012408 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2958745b-0cec-4050-8e51-420e4c11d225 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:15.012666 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2958745b-0cec-4050-8e51-420e4c11d225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:15.012931 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2958745b-0cec-4050-8e51-420e4c11d225 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:15.013554 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2958745b-0cec-4050-8e51-420e4c11d225 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:15.015168 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2958745b-0cec-4050-8e51-420e4c11d225 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:15.027798 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1242/2502] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:55:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:55:15.031850 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a3a866-e82a-4741-9c90-46825073adad admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:15.042574 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a3a866-e82a-4741-9c90-46825073adad admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:15.047412 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c041400e-1c58-401c-9378-bb500558817d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:15.048016 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041400e-1c58-401c-9378-bb500558817d admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:15.048302 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041400e-1c58-401c-9378-bb500558817d admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:15.048581 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041400e-1c58-401c-9378-bb500558817d admin admin] PATH_INFO: `/v3/projects/91a56513e9684effb2283a344d1ec12d/users/9747ddf538a34b52a8ae26688a8150ee/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:15.063669 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c041400e-1c58-401c-9378-bb500558817d admin admin] RBAC: Authorizing `identity:create_grant(project_id=91a56513e9684effb2283a344d1ec12d, user_id=9747ddf538a34b52a8ae26688a8150ee, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:15.066350 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c041400e-1c58-401c-9378-bb500558817d admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:15.153920 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1261/2503] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:55:15 2022] PUT /identity/v3/projects/91a56513e9684effb2283a344d1ec12d/users/9747ddf538a34b52a8ae26688a8150ee/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:55:15.165432 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371868b-834b-4ef6-ab10-7589b5290919 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:15.165715 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371868b-834b-4ef6-ab10-7589b5290919 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:15.165984 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371868b-834b-4ef6-ab10-7589b5290919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:15.194705 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-4371868b-834b-4ef6-ab10-7589b5290919 None None] MFA Rules not processed for user `9747ddf538a34b52a8ae26688a8150ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:55:15.195954 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4371868b-834b-4ef6-ab10-7589b5290919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:15.262563 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1243/2504] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:55:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:55:15.267117 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c041400e-1c58-401c-9378-bb500558817d admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:15.329245 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c041400e-1c58-401c-9378-bb500558817d admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:15.333937 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1631e-d8fb-4d08-931f-4771d0791dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:15.334541 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1631e-d8fb-4d08-931f-4771d0791dab admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:15.334819 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1631e-d8fb-4d08-931f-4771d0791dab admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:15.335090 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1631e-d8fb-4d08-931f-4771d0791dab admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:15.335740 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1631e-d8fb-4d08-931f-4771d0791dab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:15.337622 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1631e-d8fb-4d08-931f-4771d0791dab admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:15.457355 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1262/2505] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:55:15 2022] POST /identity/v3/projects => generated 365 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:55:15.466289 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4371868b-834b-4ef6-ab10-7589b5290919 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:15.477023 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4371868b-834b-4ef6-ab10-7589b5290919 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:15.543836 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e319b3-c7f7-472a-ab12-aadc07a421cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:15.544508 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e319b3-c7f7-472a-ab12-aadc07a421cc admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:15.544728 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e319b3-c7f7-472a-ab12-aadc07a421cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:15.545007 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e319b3-c7f7-472a-ab12-aadc07a421cc admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:15.545671 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e319b3-c7f7-472a-ab12-aadc07a421cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:15.548891 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e319b3-c7f7-472a-ab12-aadc07a421cc admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:15.645394 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1244/2506] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:55:15 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:55:15.649688 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1631e-d8fb-4d08-931f-4771d0791dab admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:15.660019 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1631e-d8fb-4d08-931f-4771d0791dab admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:15.665149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12342ee2-02fe-4b01-8579-c1817b29b74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:15.665739 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12342ee2-02fe-4b01-8579-c1817b29b74a admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:15.666017 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12342ee2-02fe-4b01-8579-c1817b29b74a admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:15.666288 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12342ee2-02fe-4b01-8579-c1817b29b74a admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:15.666927 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12342ee2-02fe-4b01-8579-c1817b29b74a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:15.668537 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12342ee2-02fe-4b01-8579-c1817b29b74a admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:15.679465 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1263/2507] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:55:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:55:15.683654 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e319b3-c7f7-472a-ab12-aadc07a421cc admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:15.693508 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e319b3-c7f7-472a-ab12-aadc07a421cc admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:15.698527 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e96afe-d7b8-469c-98bc-7adeec1aa1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:15.699180 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e96afe-d7b8-469c-98bc-7adeec1aa1e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:15.699514 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e96afe-d7b8-469c-98bc-7adeec1aa1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:15.699790 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e96afe-d7b8-469c-98bc-7adeec1aa1e0 admin admin] PATH_INFO: `/v3/projects/4fa82ffc7a8d41ba9ca79681592316a2/users/2204db5c372a4e748eea1a0caccfef8d/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:15.714841 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e96afe-d7b8-469c-98bc-7adeec1aa1e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fa82ffc7a8d41ba9ca79681592316a2, user_id=2204db5c372a4e748eea1a0caccfef8d, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:15.717595 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e96afe-d7b8-469c-98bc-7adeec1aa1e0 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:15.776970 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:55:15.782617 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1245/2508] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:55:15 2022] PUT /identity/v3/projects/4fa82ffc7a8d41ba9ca79681592316a2/users/2204db5c372a4e748eea1a0caccfef8d/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:55:15.786693 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12342ee2-02fe-4b01-8579-c1817b29b74a admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:15.797534 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12342ee2-02fe-4b01-8579-c1817b29b74a admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:15.854015 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ff465e-bdda-4a1f-a15f-0a02d5931d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:15.854901 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff465e-bdda-4a1f-a15f-0a02d5931d9a admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:15.855354 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff465e-bdda-4a1f-a15f-0a02d5931d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:15.855787 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff465e-bdda-4a1f-a15f-0a02d5931d9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:15.856766 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ff465e-bdda-4a1f-a15f-0a02d5931d9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:15.859345 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ff465e-bdda-4a1f-a15f-0a02d5931d9a admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:15.871918 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1264/2509] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:55:15 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:55:15.876215 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e96afe-d7b8-469c-98bc-7adeec1aa1e0 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:15.888317 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e96afe-d7b8-469c-98bc-7adeec1aa1e0 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:15.895614 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3e158-4e23-4655-a5c8-6ddb3501d040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:15.895614 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3e158-4e23-4655-a5c8-6ddb3501d040 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:15.895614 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3e158-4e23-4655-a5c8-6ddb3501d040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:15.895614 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3e158-4e23-4655-a5c8-6ddb3501d040 admin admin] PATH_INFO: `/v3/projects/4fa82ffc7a8d41ba9ca79681592316a2/users/2204db5c372a4e748eea1a0caccfef8d/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:15.898935 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d3e158-4e23-4655-a5c8-6ddb3501d040 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fa82ffc7a8d41ba9ca79681592316a2, user_id=2204db5c372a4e748eea1a0caccfef8d, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:15.901622 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d3e158-4e23-4655-a5c8-6ddb3501d040 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:15.912380 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1246/2510] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:55:15 2022] PUT /identity/v3/projects/4fa82ffc7a8d41ba9ca79681592316a2/users/2204db5c372a4e748eea1a0caccfef8d/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:55:15.922457 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08b76d-32c8-4e6d-b4a6-5e4027544290 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:15.922744 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08b76d-32c8-4e6d-b4a6-5e4027544290 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:15.923019 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08b76d-32c8-4e6d-b4a6-5e4027544290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:15.948739 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:55:15.951525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-2b08b76d-32c8-4e6d-b4a6-5e4027544290 None None] MFA Rules not processed for user `2204db5c372a4e748eea1a0caccfef8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:55:15.952719 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-2b08b76d-32c8-4e6d-b4a6-5e4027544290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:16.020633 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1265/2511] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:55:15 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:55:16.032276 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08b76d-32c8-4e6d-b4a6-5e4027544290 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:16.032423 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3e158-4e23-4655-a5c8-6ddb3501d040 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:16.034033 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a4d3e158-4e23-4655-a5c8-6ddb3501d040 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:16.034399 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-2b08b76d-32c8-4e6d-b4a6-5e4027544290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:16.095119 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3e158-4e23-4655-a5c8-6ddb3501d040 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:16.098738 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e675b1-e15d-46e5-8f2a-76567feae4c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:16.099371 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e675b1-e15d-46e5-8f2a-76567feae4c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:16.100249 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e675b1-e15d-46e5-8f2a-76567feae4c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:16.100518 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e675b1-e15d-46e5-8f2a-76567feae4c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:16.109768 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e675b1-e15d-46e5-8f2a-76567feae4c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:16.110362 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08b76d-32c8-4e6d-b4a6-5e4027544290 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:16.111204 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e675b1-e15d-46e5-8f2a-76567feae4c7 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:16.117305 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382d7dc7-330d-4081-a475-f06351f13ca6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:16.117914 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d7dc7-330d-4081-a475-f06351f13ca6 service nova] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:16.118193 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d7dc7-330d-4081-a475-f06351f13ca6 service nova] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:16.118758 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d7dc7-330d-4081-a475-f06351f13ca6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:16.120327 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-382d7dc7-330d-4081-a475-f06351f13ca6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:16.121632 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1247/2512] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:55:16 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:55:16.184932 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382d7dc7-330d-4081-a475-f06351f13ca6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:16.186854 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382d7dc7-330d-4081-a475-f06351f13ca6 service nova] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:16.197205 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1266/2513] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:55:16 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:55:16.438313 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e675b1-e15d-46e5-8f2a-76567feae4c7 service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:16.447301 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e675b1-e15d-46e5-8f2a-76567feae4c7 service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:16.450882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34ca18-9e3d-452c-8d60-e4d87f32ea1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:16.451506 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34ca18-9e3d-452c-8d60-e4d87f32ea1c service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:16.451784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34ca18-9e3d-452c-8d60-e4d87f32ea1c service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:16.452055 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34ca18-9e3d-452c-8d60-e4d87f32ea1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:16.510386 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b34ca18-9e3d-452c-8d60-e4d87f32ea1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:16.511874 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b34ca18-9e3d-452c-8d60-e4d87f32ea1c service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:16.520905 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1248/2514] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:55:16 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:55:22.891664 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382d7dc7-330d-4081-a475-f06351f13ca6 service nova] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:22.932417 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:55:22.956014 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382d7dc7-330d-4081-a475-f06351f13ca6 service nova] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:22.960799 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a1e67-74a7-4eaa-a259-c9dd8d2b62e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:22.961426 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a1e67-74a7-4eaa-a259-c9dd8d2b62e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:22.961762 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a1e67-74a7-4eaa-a259-c9dd8d2b62e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:22.961979 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a1e67-74a7-4eaa-a259-c9dd8d2b62e5 admin admin] PATH_INFO: `/v3/users/9747ddf538a34b52a8ae26688a8150ee` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:22.963095 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32a1e67-74a7-4eaa-a259-c9dd8d2b62e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9747ddf538a34b52a8ae26688a8150ee)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:22.965038 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32a1e67-74a7-4eaa-a259-c9dd8d2b62e5 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:23.146093 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-b32a1e67-74a7-4eaa-a259-c9dd8d2b62e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9747ddf538a34b52a8ae26688a8150ee'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:23.153002 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-b32a1e67-74a7-4eaa-a259-c9dd8d2b62e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9747ddf538a34b52a8ae26688a8150ee'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:23.165255 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-b32a1e67-74a7-4eaa-a259-c9dd8d2b62e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9747ddf538a34b52a8ae26688a8150ee'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:23.166135 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-b32a1e67-74a7-4eaa-a259-c9dd8d2b62e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9747ddf538a34b52a8ae26688a8150ee'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:23.235619 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-b32a1e67-74a7-4eaa-a259-c9dd8d2b62e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9747ddf538a34b52a8ae26688a8150ee'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:23.243860 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1267/2515] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:55:22 2022] DELETE /identity/v3/users/9747ddf538a34b52a8ae26688a8150ee => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:55:23.257036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47788838-21f5-481b-a474-cf4943dbdf19 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:23.257555 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47788838-21f5-481b-a474-cf4943dbdf19 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:23.257676 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47788838-21f5-481b-a474-cf4943dbdf19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:23.276868 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-47788838-21f5-481b-a474-cf4943dbdf19 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:55:23.278167 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-47788838-21f5-481b-a474-cf4943dbdf19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:23.338856 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1249/2516] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:55:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:55:23.353600 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a1e67-74a7-4eaa-a259-c9dd8d2b62e5 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:23.355018 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-b32a1e67-74a7-4eaa-a259-c9dd8d2b62e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:23.417067 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a1e67-74a7-4eaa-a259-c9dd8d2b62e5 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:23.420625 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b1fd10-f00f-4ec8-a8de-5d23a4cd35fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:23.421217 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1fd10-f00f-4ec8-a8de-5d23a4cd35fa service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:23.421500 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1fd10-f00f-4ec8-a8de-5d23a4cd35fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:23.421770 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1fd10-f00f-4ec8-a8de-5d23a4cd35fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:23.431055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b1fd10-f00f-4ec8-a8de-5d23a4cd35fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:23.432513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b1fd10-f00f-4ec8-a8de-5d23a4cd35fa service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:23.442254 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1268/2517] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:55:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:55:23.771910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47788838-21f5-481b-a474-cf4943dbdf19 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:23.774147 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-47788838-21f5-481b-a474-cf4943dbdf19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:23.807033 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:55:23.850949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47788838-21f5-481b-a474-cf4943dbdf19 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:23.856357 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297fdbf9-e1f9-4503-bdac-e3a87f762c90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:23.857070 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297fdbf9-e1f9-4503-bdac-e3a87f762c90 service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:23.857347 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297fdbf9-e1f9-4503-bdac-e3a87f762c90 service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:23.857616 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297fdbf9-e1f9-4503-bdac-e3a87f762c90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:23.859461 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-297fdbf9-e1f9-4503-bdac-e3a87f762c90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:23.905978 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b1fd10-f00f-4ec8-a8de-5d23a4cd35fa service neutron] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:23.907567 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-78b1fd10-f00f-4ec8-a8de-5d23a4cd35fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:23.924824 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297fdbf9-e1f9-4503-bdac-e3a87f762c90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:23.926709 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297fdbf9-e1f9-4503-bdac-e3a87f762c90 service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:23.931646 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b1fd10-f00f-4ec8-a8de-5d23a4cd35fa service neutron] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:23.936821 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372640d0-73d0-4ae0-a371-872938455429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:23.937451 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372640d0-73d0-4ae0-a371-872938455429 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:23.937849 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1250/2518] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:55:23 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:55:23.937979 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372640d0-73d0-4ae0-a371-872938455429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:23.938180 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372640d0-73d0-4ae0-a371-872938455429 admin admin] PATH_INFO: `/v3/projects/91a56513e9684effb2283a344d1ec12d` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:23.939410 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372640d0-73d0-4ae0-a371-872938455429 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91a56513e9684effb2283a344d1ec12d)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:23.941326 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372640d0-73d0-4ae0-a371-872938455429 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:24.055813 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-372640d0-73d0-4ae0-a371-872938455429 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91a56513e9684effb2283a344d1ec12d', 'request_id': 'req-372640d0-73d0-4ae0-a371-872938455429'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:24.057866 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:55:24.063556 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-372640d0-73d0-4ae0-a371-872938455429 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91a56513e9684effb2283a344d1ec12d', 'request_id': 'req-372640d0-73d0-4ae0-a371-872938455429'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:24.118612 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-372640d0-73d0-4ae0-a371-872938455429 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91a56513e9684effb2283a344d1ec12d', 'request_id': 'req-372640d0-73d0-4ae0-a371-872938455429'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:24.129495 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-372640d0-73d0-4ae0-a371-872938455429 admin admin] The token cache is being invalidate because project 91a56513e9684effb2283a344d1ec12d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:55:24.129778 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-372640d0-73d0-4ae0-a371-872938455429 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:24.130927 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-372640d0-73d0-4ae0-a371-872938455429 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:24.132470 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1269/2519] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:55:23 2022] DELETE /identity/v3/projects/91a56513e9684effb2283a344d1ec12d => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:55:24.136823 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297fdbf9-e1f9-4503-bdac-e3a87f762c90 service nova] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:24.150368 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297fdbf9-e1f9-4503-bdac-e3a87f762c90 service nova] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:24.205593 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb5f5c-dda4-494f-b0d2-9616d68445bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:24.206229 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb5f5c-dda4-494f-b0d2-9616d68445bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:24.206506 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb5f5c-dda4-494f-b0d2-9616d68445bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:24.206805 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb5f5c-dda4-494f-b0d2-9616d68445bc admin admin] PATH_INFO: `/v3/users/2204db5c372a4e748eea1a0caccfef8d` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:24.207933 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb5f5c-dda4-494f-b0d2-9616d68445bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=2204db5c372a4e748eea1a0caccfef8d)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:24.210788 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb5f5c-dda4-494f-b0d2-9616d68445bc admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:24.509378 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-0afb5f5c-dda4-494f-b0d2-9616d68445bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2204db5c372a4e748eea1a0caccfef8d'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:24.522413 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-0afb5f5c-dda4-494f-b0d2-9616d68445bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2204db5c372a4e748eea1a0caccfef8d'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:24.523159 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-0afb5f5c-dda4-494f-b0d2-9616d68445bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2204db5c372a4e748eea1a0caccfef8d'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:24.524580 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:55:24.531609 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-0afb5f5c-dda4-494f-b0d2-9616d68445bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2204db5c372a4e748eea1a0caccfef8d'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:24.607269 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-0afb5f5c-dda4-494f-b0d2-9616d68445bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2204db5c372a4e748eea1a0caccfef8d'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:24.617002 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1251/2520] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:55:24 2022] DELETE /identity/v3/users/2204db5c372a4e748eea1a0caccfef8d => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:55:25.080421 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372640d0-73d0-4ae0-a371-872938455429 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:25.082031 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-372640d0-73d0-4ae0-a371-872938455429 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:25.095594 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372640d0-73d0-4ae0-a371-872938455429 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:25.155177 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12b07c-4da8-450e-87fc-40819d56ea3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:25.155875 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12b07c-4da8-450e-87fc-40819d56ea3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:25.156160 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12b07c-4da8-450e-87fc-40819d56ea3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:25.156435 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12b07c-4da8-450e-87fc-40819d56ea3a admin admin] PATH_INFO: `/v3/projects/4fa82ffc7a8d41ba9ca79681592316a2` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:25.157582 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa12b07c-4da8-450e-87fc-40819d56ea3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fa82ffc7a8d41ba9ca79681592316a2)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:25.159529 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa12b07c-4da8-450e-87fc-40819d56ea3a admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:25.254238 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-aa12b07c-4da8-450e-87fc-40819d56ea3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fa82ffc7a8d41ba9ca79681592316a2', 'request_id': 'req-aa12b07c-4da8-450e-87fc-40819d56ea3a'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:25.260934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-aa12b07c-4da8-450e-87fc-40819d56ea3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fa82ffc7a8d41ba9ca79681592316a2', 'request_id': 'req-aa12b07c-4da8-450e-87fc-40819d56ea3a'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:25.344389 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-aa12b07c-4da8-450e-87fc-40819d56ea3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fa82ffc7a8d41ba9ca79681592316a2', 'request_id': 'req-aa12b07c-4da8-450e-87fc-40819d56ea3a'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:25.356347 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-aa12b07c-4da8-450e-87fc-40819d56ea3a admin admin] The token cache is being invalidate because project 4fa82ffc7a8d41ba9ca79681592316a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:55:25.356636 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-aa12b07c-4da8-450e-87fc-40819d56ea3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:25.357278 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-aa12b07c-4da8-450e-87fc-40819d56ea3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:25.358793 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1270/2521] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:55:25 2022] DELETE /identity/v3/projects/4fa82ffc7a8d41ba9ca79681592316a2 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:55:25.363717 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f208ab-f3fc-4077-8c97-66d36811d772 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:25.364036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f208ab-f3fc-4077-8c97-66d36811d772 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:25.364312 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f208ab-f3fc-4077-8c97-66d36811d772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:25.387581 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-a5f208ab-f3fc-4077-8c97-66d36811d772 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:55:25.389474 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a5f208ab-f3fc-4077-8c97-66d36811d772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:25.452909 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1252/2522] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:55:25.462535 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e9c1e-4588-4dcd-8386-8c7569744e0e None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:25.462819 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e9c1e-4588-4dcd-8386-8c7569744e0e None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:25.463116 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e9c1e-4588-4dcd-8386-8c7569744e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:25.482270 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-3c7e9c1e-4588-4dcd-8386-8c7569744e0e None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:55:25.483493 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-3c7e9c1e-4588-4dcd-8386-8c7569744e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:25.491610 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1271/2523] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:55:25.495487 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f208ab-f3fc-4077-8c97-66d36811d772 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:25.505066 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f208ab-f3fc-4077-8c97-66d36811d772 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:25.510375 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837859de-cbcc-464b-ade4-5e9fa5faf52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:25.510998 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837859de-cbcc-464b-ade4-5e9fa5faf52c admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:25.511240 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837859de-cbcc-464b-ade4-5e9fa5faf52c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:25.511550 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837859de-cbcc-464b-ade4-5e9fa5faf52c admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:25.512190 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837859de-cbcc-464b-ade4-5e9fa5faf52c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:55:25.512492 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837859de-cbcc-464b-ade4-5e9fa5faf52c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:25.514069 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837859de-cbcc-464b-ade4-5e9fa5faf52c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:25.525867 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1253/2524] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:55:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:55:25.530135 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e9c1e-4588-4dcd-8386-8c7569744e0e None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:25.539141 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e9c1e-4588-4dcd-8386-8c7569744e0e None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:25.543743 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb4554-53e2-4dfd-9461-9c8323070f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:25.544334 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb4554-53e2-4dfd-9461-9c8323070f06 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:25.544611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb4554-53e2-4dfd-9461-9c8323070f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:25.544881 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb4554-53e2-4dfd-9461-9c8323070f06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:25.545492 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffb4554-53e2-4dfd-9461-9c8323070f06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:25.547386 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffb4554-53e2-4dfd-9461-9c8323070f06 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:25.634874 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1272/2525] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:55:25 2022] POST /identity/v3/projects => generated 361 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:55:25.642188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837859de-cbcc-464b-ade4-5e9fa5faf52c admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:25.650579 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837859de-cbcc-464b-ade4-5e9fa5faf52c admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:25.708022 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18a43e7-c7e1-4def-9e17-427a70284105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:25.709238 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a43e7-c7e1-4def-9e17-427a70284105 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:25.710011 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a43e7-c7e1-4def-9e17-427a70284105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:25.710011 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a43e7-c7e1-4def-9e17-427a70284105 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:25.710632 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18a43e7-c7e1-4def-9e17-427a70284105 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:25.712651 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18a43e7-c7e1-4def-9e17-427a70284105 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:25.826616 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1254/2526] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:55:25 2022] POST /identity/v3/users => generated 281 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:55:25.830748 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb4554-53e2-4dfd-9461-9c8323070f06 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:25.841506 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb4554-53e2-4dfd-9461-9c8323070f06 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:25.846243 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187a88e-fac1-430c-8a92-362838a7cbbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:25.846781 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187a88e-fac1-430c-8a92-362838a7cbbc admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:25.847059 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187a88e-fac1-430c-8a92-362838a7cbbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:25.847367 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187a88e-fac1-430c-8a92-362838a7cbbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:25.848011 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4187a88e-fac1-430c-8a92-362838a7cbbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:25.849665 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4187a88e-fac1-430c-8a92-362838a7cbbc admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:25.860912 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1273/2527] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 12:55:25.865878 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18a43e7-c7e1-4def-9e17-427a70284105 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:25.877172 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18a43e7-c7e1-4def-9e17-427a70284105 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:25.882583 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce90a66a-802a-4a36-a6e0-6785b5c484b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:25.883114 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90a66a-802a-4a36-a6e0-6785b5c484b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:25.883428 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90a66a-802a-4a36-a6e0-6785b5c484b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:25.883743 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90a66a-802a-4a36-a6e0-6785b5c484b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:25.884354 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce90a66a-802a-4a36-a6e0-6785b5c484b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:25.885923 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce90a66a-802a-4a36-a6e0-6785b5c484b4 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:25.896527 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1255/2528] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:55:25.901178 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187a88e-fac1-430c-8a92-362838a7cbbc admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:25.910868 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187a88e-fac1-430c-8a92-362838a7cbbc admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:25.915580 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d142e-bf5e-446e-a3ce-e8d93fb4c58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:25.916223 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d142e-bf5e-446e-a3ce-e8d93fb4c58f admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:25.916500 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d142e-bf5e-446e-a3ce-e8d93fb4c58f admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:25.916769 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d142e-bf5e-446e-a3ce-e8d93fb4c58f admin admin] PATH_INFO: `/v3/projects/2398d20943e7411bb80812bd4048c49e/users/84f71d9d5293472b81818d941072e783/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:25.932055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011d142e-bf5e-446e-a3ce-e8d93fb4c58f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2398d20943e7411bb80812bd4048c49e, user_id=84f71d9d5293472b81818d941072e783, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:25.937833 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011d142e-bf5e-446e-a3ce-e8d93fb4c58f admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:26.019343 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1274/2529] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:55:25 2022] PUT /identity/v3/projects/2398d20943e7411bb80812bd4048c49e/users/84f71d9d5293472b81818d941072e783/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:55:26.030470 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc2869-796e-47fd-b635-42d1a126393d None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:26.030755 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc2869-796e-47fd-b635-42d1a126393d None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:26.031029 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc2869-796e-47fd-b635-42d1a126393d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:26.062508 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-18bc2869-796e-47fd-b635-42d1a126393d None None] MFA Rules not processed for user `84f71d9d5293472b81818d941072e783`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:55:26.063886 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-18bc2869-796e-47fd-b635-42d1a126393d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:26.104900 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:55:26.131594 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1256/2530] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:55:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:55:26.166433 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d142e-bf5e-446e-a3ce-e8d93fb4c58f admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:26.171001 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-011d142e-bf5e-446e-a3ce-e8d93fb4c58f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:26.178596 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:55:26.226144 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d142e-bf5e-446e-a3ce-e8d93fb4c58f admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:26.229854 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2888bb1a-af96-4070-a060-71b5cc248d2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:26.230454 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888bb1a-af96-4070-a060-71b5cc248d2f service cinder] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:26.230733 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888bb1a-af96-4070-a060-71b5cc248d2f service cinder] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:26.231007 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888bb1a-af96-4070-a060-71b5cc248d2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:26.239712 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2888bb1a-af96-4070-a060-71b5cc248d2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:26.241969 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2888bb1a-af96-4070-a060-71b5cc248d2f service cinder] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:26.251152 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1275/2531] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:55:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:55:26.267418 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc2869-796e-47fd-b635-42d1a126393d None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:26.334296 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc2869-796e-47fd-b635-42d1a126393d None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:26.338890 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55150e-36cb-4c4f-aed1-b7b6081eb147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:26.339546 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55150e-36cb-4c4f-aed1-b7b6081eb147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:26.339829 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55150e-36cb-4c4f-aed1-b7b6081eb147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:26.340102 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55150e-36cb-4c4f-aed1-b7b6081eb147 admin admin] PATH_INFO: `/v3/users/84f71d9d5293472b81818d941072e783` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:26.341220 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd55150e-36cb-4c4f-aed1-b7b6081eb147 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84f71d9d5293472b81818d941072e783)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:26.343126 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd55150e-36cb-4c4f-aed1-b7b6081eb147 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:26.568534 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-cd55150e-36cb-4c4f-aed1-b7b6081eb147 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84f71d9d5293472b81818d941072e783'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:26.588317 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-cd55150e-36cb-4c4f-aed1-b7b6081eb147 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84f71d9d5293472b81818d941072e783'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:26.588628 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-cd55150e-36cb-4c4f-aed1-b7b6081eb147 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84f71d9d5293472b81818d941072e783'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:26.601874 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-cd55150e-36cb-4c4f-aed1-b7b6081eb147 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84f71d9d5293472b81818d941072e783'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:26.675490 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-cd55150e-36cb-4c4f-aed1-b7b6081eb147 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84f71d9d5293472b81818d941072e783'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:26.690001 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1257/2532] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:55:26 2022] DELETE /identity/v3/users/84f71d9d5293472b81818d941072e783 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:55:26.695256 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9099a5-737c-41fb-b5f4-bf555af6ed5b None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:26.695780 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9099a5-737c-41fb-b5f4-bf555af6ed5b None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:26.695975 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9099a5-737c-41fb-b5f4-bf555af6ed5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:26.715023 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-aa9099a5-737c-41fb-b5f4-bf555af6ed5b None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:55:26.716262 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-aa9099a5-737c-41fb-b5f4-bf555af6ed5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:26.775915 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1276/2533] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:55:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:55:26.785864 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55150e-36cb-4c4f-aed1-b7b6081eb147 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:26.787453 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-cd55150e-36cb-4c4f-aed1-b7b6081eb147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:26.846610 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55150e-36cb-4c4f-aed1-b7b6081eb147 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:26.851369 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3decae8-a17b-4ee5-b01f-59e0f7c9108d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:26.851997 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3decae8-a17b-4ee5-b01f-59e0f7c9108d service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:26.853223 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3decae8-a17b-4ee5-b01f-59e0f7c9108d service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:26.853223 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3decae8-a17b-4ee5-b01f-59e0f7c9108d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:26.862122 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3decae8-a17b-4ee5-b01f-59e0f7c9108d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:26.863612 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3decae8-a17b-4ee5-b01f-59e0f7c9108d service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:26.869919 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:55:26.875212 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1258/2534] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:55:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:55:27.299938 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9099a5-737c-41fb-b5f4-bf555af6ed5b None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:27.301457 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-aa9099a5-737c-41fb-b5f4-bf555af6ed5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:27.313743 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9099a5-737c-41fb-b5f4-bf555af6ed5b None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:27.318814 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271c62e2-ae45-4386-8be9-5534a9b93bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:27.319683 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c62e2-ae45-4386-8be9-5534a9b93bad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:27.319968 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c62e2-ae45-4386-8be9-5534a9b93bad admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:27.320242 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c62e2-ae45-4386-8be9-5534a9b93bad admin admin] PATH_INFO: `/v3/projects/2398d20943e7411bb80812bd4048c49e` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:27.321483 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271c62e2-ae45-4386-8be9-5534a9b93bad admin admin] RBAC: Authorizing `identity:delete_project(project_id=2398d20943e7411bb80812bd4048c49e)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:27.323471 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271c62e2-ae45-4386-8be9-5534a9b93bad admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:27.425027 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-271c62e2-ae45-4386-8be9-5534a9b93bad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2398d20943e7411bb80812bd4048c49e', 'request_id': 'req-271c62e2-ae45-4386-8be9-5534a9b93bad'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:27.431448 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-271c62e2-ae45-4386-8be9-5534a9b93bad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2398d20943e7411bb80812bd4048c49e', 'request_id': 'req-271c62e2-ae45-4386-8be9-5534a9b93bad'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:27.515725 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-271c62e2-ae45-4386-8be9-5534a9b93bad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2398d20943e7411bb80812bd4048c49e', 'request_id': 'req-271c62e2-ae45-4386-8be9-5534a9b93bad'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:27.527663 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-271c62e2-ae45-4386-8be9-5534a9b93bad admin admin] The token cache is being invalidate because project 2398d20943e7411bb80812bd4048c49e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:55:27.527958 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-271c62e2-ae45-4386-8be9-5534a9b93bad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:27.528601 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-271c62e2-ae45-4386-8be9-5534a9b93bad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:27.530231 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1277/2535] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:55:27 2022] DELETE /identity/v3/projects/2398d20943e7411bb80812bd4048c49e => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:55:27.535336 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ee77c-53d5-4590-9e68-1c692bf54f22 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:27.535624 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ee77c-53d5-4590-9e68-1c692bf54f22 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:27.535900 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ee77c-53d5-4590-9e68-1c692bf54f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:27.556114 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-f43ee77c-53d5-4590-9e68-1c692bf54f22 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:55:27.557312 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-f43ee77c-53d5-4590-9e68-1c692bf54f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:27.615822 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1259/2536] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:55:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:55:27.623749 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25701e68-20b7-4d17-bc1b-35cad068beb8 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:27.629290 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25701e68-20b7-4d17-bc1b-35cad068beb8 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:27.629290 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25701e68-20b7-4d17-bc1b-35cad068beb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:27.648831 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-25701e68-20b7-4d17-bc1b-35cad068beb8 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:55:27.650083 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-25701e68-20b7-4d17-bc1b-35cad068beb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:27.658215 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1278/2537] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:55:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:55:27.662716 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43ee77c-53d5-4590-9e68-1c692bf54f22 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:27.672433 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43ee77c-53d5-4590-9e68-1c692bf54f22 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:27.677065 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c9432-2d50-4efe-b4c9-0acb2fc695ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:27.677963 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c9432-2d50-4efe-b4c9-0acb2fc695ff admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:27.678307 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c9432-2d50-4efe-b4c9-0acb2fc695ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:27.678593 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c9432-2d50-4efe-b4c9-0acb2fc695ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:27.679265 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c9432-2d50-4efe-b4c9-0acb2fc695ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:55:27.679603 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c9432-2d50-4efe-b4c9-0acb2fc695ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:27.682036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c9432-2d50-4efe-b4c9-0acb2fc695ff admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:27.697462 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1260/2538] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:55:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:55:27.698637 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25701e68-20b7-4d17-bc1b-35cad068beb8 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:27.709272 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25701e68-20b7-4d17-bc1b-35cad068beb8 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:27.714288 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f1139-1759-4fc8-8e61-a9345550f111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:27.714911 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f1139-1759-4fc8-8e61-a9345550f111 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:27.715190 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f1139-1759-4fc8-8e61-a9345550f111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:27.715504 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f1139-1759-4fc8-8e61-a9345550f111 admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:27.716190 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51f1139-1759-4fc8-8e61-a9345550f111 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:27.718172 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51f1139-1759-4fc8-8e61-a9345550f111 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:27.837387 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1279/2539] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:55:27 2022] POST /identity/v3/projects => generated 375 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:55:27.837816 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c9432-2d50-4efe-b4c9-0acb2fc695ff admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:27.840368 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c9432-2d50-4efe-b4c9-0acb2fc695ff admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:27.901309 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4188a551-435e-4f37-8855-c9445aac0a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:27.901921 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188a551-435e-4f37-8855-c9445aac0a2b admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:27.902200 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188a551-435e-4f37-8855-c9445aac0a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:27.902475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188a551-435e-4f37-8855-c9445aac0a2b admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:27.903102 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4188a551-435e-4f37-8855-c9445aac0a2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:27.905024 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4188a551-435e-4f37-8855-c9445aac0a2b admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:27.990369 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1261/2540] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:55:27 2022] POST /identity/v3/users => generated 288 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:55:27.995023 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f1139-1759-4fc8-8e61-a9345550f111 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:28.001704 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:55:28.005674 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f1139-1759-4fc8-8e61-a9345550f111 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:28.010467 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370e071-41ae-47c2-be22-470643124d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:28.016510 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370e071-41ae-47c2-be22-470643124d7e admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:28.016510 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370e071-41ae-47c2-be22-470643124d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:28.016510 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370e071-41ae-47c2-be22-470643124d7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:28.016510 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8370e071-41ae-47c2-be22-470643124d7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:28.016510 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8370e071-41ae-47c2-be22-470643124d7e admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:28.028154 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1280/2541] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:55:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:55:28.029081 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4188a551-435e-4f37-8855-c9445aac0a2b admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:28.039972 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4188a551-435e-4f37-8855-c9445aac0a2b admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:28.044909 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684873e7-028c-440c-9e45-8e2002534660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:28.050009 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684873e7-028c-440c-9e45-8e2002534660 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:28.050009 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684873e7-028c-440c-9e45-8e2002534660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:28.050009 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684873e7-028c-440c-9e45-8e2002534660 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:28.050009 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684873e7-028c-440c-9e45-8e2002534660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:28.050009 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684873e7-028c-440c-9e45-8e2002534660 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:28.063697 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1262/2542] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:55:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:55:28.064570 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370e071-41ae-47c2-be22-470643124d7e admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:28.084293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370e071-41ae-47c2-be22-470643124d7e admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:28.084293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805e5d4-db5c-4384-9f0d-2e0db1731276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:28.084293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805e5d4-db5c-4384-9f0d-2e0db1731276 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:28.084293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805e5d4-db5c-4384-9f0d-2e0db1731276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:28.084762 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805e5d4-db5c-4384-9f0d-2e0db1731276 admin admin] PATH_INFO: `/v3/projects/c5cc8219e0b54974bdfa22493daa0ff0/users/4e302e5dc3ec41739c191f02dc083ab5/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:28.100877 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7805e5d4-db5c-4384-9f0d-2e0db1731276 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5cc8219e0b54974bdfa22493daa0ff0, user_id=4e302e5dc3ec41739c191f02dc083ab5, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:28.103740 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7805e5d4-db5c-4384-9f0d-2e0db1731276 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:28.213566 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1281/2543] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:55:28 2022] PUT /identity/v3/projects/c5cc8219e0b54974bdfa22493daa0ff0/users/4e302e5dc3ec41739c191f02dc083ab5/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:55:28.224888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6819d-ca0c-48a2-a91f-b38bb33610d1 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:28.225244 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6819d-ca0c-48a2-a91f-b38bb33610d1 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:28.225584 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6819d-ca0c-48a2-a91f-b38bb33610d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:28.256288 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-4da6819d-ca0c-48a2-a91f-b38bb33610d1 None None] MFA Rules not processed for user `4e302e5dc3ec41739c191f02dc083ab5`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:55:28.257512 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4da6819d-ca0c-48a2-a91f-b38bb33610d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:28.317504 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:55:28.327507 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1263/2544] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:55:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:55:28.332476 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805e5d4-db5c-4384-9f0d-2e0db1731276 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:28.405184 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805e5d4-db5c-4384-9f0d-2e0db1731276 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:28.409982 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda792c8-5c9b-4846-838b-e4028bf90d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:28.410598 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda792c8-5c9b-4846-838b-e4028bf90d37 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:28.410883 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda792c8-5c9b-4846-838b-e4028bf90d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:28.411155 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda792c8-5c9b-4846-838b-e4028bf90d37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:28.412247 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda792c8-5c9b-4846-838b-e4028bf90d37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:28.414120 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda792c8-5c9b-4846-838b-e4028bf90d37 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:28.522572 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1282/2545] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:55:28 2022] POST /identity/v3/projects => generated 373 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:55:28.526821 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da6819d-ca0c-48a2-a91f-b38bb33610d1 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:28.538649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da6819d-ca0c-48a2-a91f-b38bb33610d1 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:28.594271 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7dccc-7a34-4bd6-a142-3e76f36c51ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:28.594884 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7dccc-7a34-4bd6-a142-3e76f36c51ca admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:28.595159 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7dccc-7a34-4bd6-a142-3e76f36c51ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:28.595463 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7dccc-7a34-4bd6-a142-3e76f36c51ca admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:28.596093 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e7dccc-7a34-4bd6-a142-3e76f36c51ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:28.597986 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e7dccc-7a34-4bd6-a142-3e76f36c51ca admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:28.715059 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1264/2546] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:55:28 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:55:28.718994 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda792c8-5c9b-4846-838b-e4028bf90d37 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:28.730113 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda792c8-5c9b-4846-838b-e4028bf90d37 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:28.734783 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f0d8d-4d02-499c-97e1-466f0fd53a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:28.735431 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f0d8d-4d02-499c-97e1-466f0fd53a37 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:28.740755 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f0d8d-4d02-499c-97e1-466f0fd53a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:28.740755 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f0d8d-4d02-499c-97e1-466f0fd53a37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:28.740755 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f0d8d-4d02-499c-97e1-466f0fd53a37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:28.740755 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f0d8d-4d02-499c-97e1-466f0fd53a37 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:28.755107 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1283/2547] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:55:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 12:55:28.756666 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7dccc-7a34-4bd6-a142-3e76f36c51ca admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:28.767835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7dccc-7a34-4bd6-a142-3e76f36c51ca admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:28.775295 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fbf12f-e00b-42a3-a461-e5de6617b9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:28.776570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fbf12f-e00b-42a3-a461-e5de6617b9a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:28.777034 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fbf12f-e00b-42a3-a461-e5de6617b9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:28.777570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fbf12f-e00b-42a3-a461-e5de6617b9a1 admin admin] PATH_INFO: `/v3/projects/e4d478efd46a45f3827613c13b9e2480/users/191a771b2f214de991bec219d974fbfb/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:28.794818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fbf12f-e00b-42a3-a461-e5de6617b9a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4d478efd46a45f3827613c13b9e2480, user_id=191a771b2f214de991bec219d974fbfb, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:28.797690 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fbf12f-e00b-42a3-a461-e5de6617b9a1 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:28.862528 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1265/2548] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:55:28 2022] PUT /identity/v3/projects/e4d478efd46a45f3827613c13b9e2480/users/191a771b2f214de991bec219d974fbfb/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:55:28.867337 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f0d8d-4d02-499c-97e1-466f0fd53a37 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:28.877653 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f0d8d-4d02-499c-97e1-466f0fd53a37 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:28.952944 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1117bc-ec91-4e09-a19f-2d6a2426c988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:28.953706 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1117bc-ec91-4e09-a19f-2d6a2426c988 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:28.953997 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1117bc-ec91-4e09-a19f-2d6a2426c988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:28.954267 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1117bc-ec91-4e09-a19f-2d6a2426c988 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:28.955097 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1117bc-ec91-4e09-a19f-2d6a2426c988 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:28.956871 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1117bc-ec91-4e09-a19f-2d6a2426c988 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:28.978889 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1284/2549] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:55:28 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:55:28.987769 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fbf12f-e00b-42a3-a461-e5de6617b9a1 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:28.999850 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fbf12f-e00b-42a3-a461-e5de6617b9a1 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:29.004791 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83562f2-a13d-4509-9a82-21d1617e3afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:29.005553 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83562f2-a13d-4509-9a82-21d1617e3afc admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:29.006487 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83562f2-a13d-4509-9a82-21d1617e3afc admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:29.006772 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83562f2-a13d-4509-9a82-21d1617e3afc admin admin] PATH_INFO: `/v3/projects/e4d478efd46a45f3827613c13b9e2480/users/191a771b2f214de991bec219d974fbfb/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:29.008856 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83562f2-a13d-4509-9a82-21d1617e3afc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4d478efd46a45f3827613c13b9e2480, user_id=191a771b2f214de991bec219d974fbfb, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:29.011856 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83562f2-a13d-4509-9a82-21d1617e3afc admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:29.030991 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1266/2550] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:55:28 2022] PUT /identity/v3/projects/e4d478efd46a45f3827613c13b9e2480/users/191a771b2f214de991bec219d974fbfb/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:55:29.042149 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1f4ba-8b67-4980-83a9-29698d4fd7ae None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:29.043103 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1f4ba-8b67-4980-83a9-29698d4fd7ae None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:29.043103 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1f4ba-8b67-4980-83a9-29698d4fd7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:29.076614 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-adb1f4ba-8b67-4980-83a9-29698d4fd7ae None None] MFA Rules not processed for user `191a771b2f214de991bec219d974fbfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:55:29.077995 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-adb1f4ba-8b67-4980-83a9-29698d4fd7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:29.113096 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:55:29.175825 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1285/2551] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:55:29 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:55:29.188977 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83562f2-a13d-4509-9a82-21d1617e3afc admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:29.191787 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b83562f2-a13d-4509-9a82-21d1617e3afc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:29.277962 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83562f2-a13d-4509-9a82-21d1617e3afc admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:29.282090 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc94f465-093b-4c8f-899e-5a410135f853 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:29.282841 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94f465-093b-4c8f-899e-5a410135f853 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:29.283168 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94f465-093b-4c8f-899e-5a410135f853 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:29.283811 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94f465-093b-4c8f-899e-5a410135f853 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:29.366495 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc94f465-093b-4c8f-899e-5a410135f853 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:29.368004 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc94f465-093b-4c8f-899e-5a410135f853 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:29.385376 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1267/2552] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:55:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:55:31.162568 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aaa984-7b7b-4edf-b35f-c3e097aa5431 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:31.163046 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aaa984-7b7b-4edf-b35f-c3e097aa5431 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:31.163158 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aaa984-7b7b-4edf-b35f-c3e097aa5431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:31.181883 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-98aaa984-7b7b-4edf-b35f-c3e097aa5431 None None] MFA Rules not processed for user `7ffffa27986f461ebc10c23c3c9817f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:55:31.183067 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-98aaa984-7b7b-4edf-b35f-c3e097aa5431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:31.242626 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1286/2553] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:55:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:55:31.564071 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc94f465-093b-4c8f-899e-5a410135f853 service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:31.565534 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-bc94f465-093b-4c8f-899e-5a410135f853 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:31.570831 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:55:31.630010 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc94f465-093b-4c8f-899e-5a410135f853 service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:31.633663 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea4cb84-4157-4a70-9f3c-8af97b150ff1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:31.634279 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea4cb84-4157-4a70-9f3c-8af97b150ff1 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:31.634827 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea4cb84-4157-4a70-9f3c-8af97b150ff1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:31.635106 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea4cb84-4157-4a70-9f3c-8af97b150ff1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:31.637365 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-fea4cb84-4157-4a70-9f3c-8af97b150ff1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:31.647544 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea4cb84-4157-4a70-9f3c-8af97b150ff1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:31.650458 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea4cb84-4157-4a70-9f3c-8af97b150ff1 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:31.657735 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1268/2554] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:55:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:55:32.374627 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aaa984-7b7b-4edf-b35f-c3e097aa5431 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:32.376035 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-98aaa984-7b7b-4edf-b35f-c3e097aa5431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:32.447093 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aaa984-7b7b-4edf-b35f-c3e097aa5431 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:32.452293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a196acf-9679-4c1c-b91f-3052dca6e103 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25eef64e92e9422ea62508c3eb0a197e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:32.453257 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a196acf-9679-4c1c-b91f-3052dca6e103 service placement] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:32.453539 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a196acf-9679-4c1c-b91f-3052dca6e103 service placement] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:32.453803 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a196acf-9679-4c1c-b91f-3052dca6e103 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:32.455617 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-4a196acf-9679-4c1c-b91f-3052dca6e103 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:32.467365 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a196acf-9679-4c1c-b91f-3052dca6e103 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:32.469008 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a196acf-9679-4c1c-b91f-3052dca6e103 service placement] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:32.472581 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea4cb84-4157-4a70-9f3c-8af97b150ff1 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:32.479493 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1287/2555] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:55:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:55:32.537424 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea4cb84-4157-4a70-9f3c-8af97b150ff1 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:32.542126 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1fae64-e47c-459d-8730-e45900cc215c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:32.542782 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1fae64-e47c-459d-8730-e45900cc215c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:32.543079 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1fae64-e47c-459d-8730-e45900cc215c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:32.543395 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1fae64-e47c-459d-8730-e45900cc215c admin admin] PATH_INFO: `/v3/users/4e302e5dc3ec41739c191f02dc083ab5` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:32.544516 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1fae64-e47c-459d-8730-e45900cc215c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e302e5dc3ec41739c191f02dc083ab5)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:32.546415 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1fae64-e47c-459d-8730-e45900cc215c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:32.751466 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1a1fae64-e47c-459d-8730-e45900cc215c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e302e5dc3ec41739c191f02dc083ab5'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:32.764503 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1a1fae64-e47c-459d-8730-e45900cc215c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e302e5dc3ec41739c191f02dc083ab5'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:32.765183 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1a1fae64-e47c-459d-8730-e45900cc215c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e302e5dc3ec41739c191f02dc083ab5'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:32.773027 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1a1fae64-e47c-459d-8730-e45900cc215c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e302e5dc3ec41739c191f02dc083ab5'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:32.867690 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-1a1fae64-e47c-459d-8730-e45900cc215c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e302e5dc3ec41739c191f02dc083ab5'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:32.874012 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1269/2556] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:55:32 2022] DELETE /identity/v3/users/4e302e5dc3ec41739c191f02dc083ab5 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:55:32.878619 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd1fc8-5bd0-4742-8243-0fdba41b31ee None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:32.878995 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd1fc8-5bd0-4742-8243-0fdba41b31ee None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:32.879445 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd1fc8-5bd0-4742-8243-0fdba41b31ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:32.902123 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-18cd1fc8-5bd0-4742-8243-0fdba41b31ee None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:55:32.903372 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-18cd1fc8-5bd0-4742-8243-0fdba41b31ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:32.960772 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:55:32.969675 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1288/2557] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:55:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:55:32.979919 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1fae64-e47c-459d-8730-e45900cc215c admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:32.981384 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1a1fae64-e47c-459d-8730-e45900cc215c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:33.015005 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:55:33.063085 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1fae64-e47c-459d-8730-e45900cc215c admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:33.068013 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a1f9f-2531-435d-9589-000681e885f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:33.068490 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a1f9f-2531-435d-9589-000681e885f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:33.068771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a1f9f-2531-435d-9589-000681e885f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:33.069226 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a1f9f-2531-435d-9589-000681e885f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:33.080246 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a1f9f-2531-435d-9589-000681e885f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:33.081590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a1f9f-2531-435d-9589-000681e885f4 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:33.092749 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1270/2558] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:55:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:55:33.596314 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd1fc8-5bd0-4742-8243-0fdba41b31ee None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:33.597737 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-18cd1fc8-5bd0-4742-8243-0fdba41b31ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:33.610270 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd1fc8-5bd0-4742-8243-0fdba41b31ee None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:33.615562 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431cfb42-ff84-4dd4-80c7-43eae86d46e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:33.616486 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431cfb42-ff84-4dd4-80c7-43eae86d46e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:33.616765 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431cfb42-ff84-4dd4-80c7-43eae86d46e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:33.617037 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431cfb42-ff84-4dd4-80c7-43eae86d46e9 admin admin] PATH_INFO: `/v3/projects/c5cc8219e0b54974bdfa22493daa0ff0` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:33.618168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431cfb42-ff84-4dd4-80c7-43eae86d46e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5cc8219e0b54974bdfa22493daa0ff0)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:33.629157 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431cfb42-ff84-4dd4-80c7-43eae86d46e9 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:33.759281 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-431cfb42-ff84-4dd4-80c7-43eae86d46e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5cc8219e0b54974bdfa22493daa0ff0', 'request_id': 'req-431cfb42-ff84-4dd4-80c7-43eae86d46e9'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:33.765505 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-431cfb42-ff84-4dd4-80c7-43eae86d46e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5cc8219e0b54974bdfa22493daa0ff0', 'request_id': 'req-431cfb42-ff84-4dd4-80c7-43eae86d46e9'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:33.890122 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-431cfb42-ff84-4dd4-80c7-43eae86d46e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5cc8219e0b54974bdfa22493daa0ff0', 'request_id': 'req-431cfb42-ff84-4dd4-80c7-43eae86d46e9'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:33.902124 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-431cfb42-ff84-4dd4-80c7-43eae86d46e9 admin admin] The token cache is being invalidate because project c5cc8219e0b54974bdfa22493daa0ff0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:55:33.902437 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-431cfb42-ff84-4dd4-80c7-43eae86d46e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:33.903065 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-431cfb42-ff84-4dd4-80c7-43eae86d46e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:33.904601 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1289/2559] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:55:33 2022] DELETE /identity/v3/projects/c5cc8219e0b54974bdfa22493daa0ff0 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:55:33.908845 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a1f9f-2531-435d-9589-000681e885f4 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:33.920300 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a1f9f-2531-435d-9589-000681e885f4 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:33.976918 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dd8330-a6d0-47f6-9187-5b68af6354e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:33.977537 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd8330-a6d0-47f6-9187-5b68af6354e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:33.977814 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd8330-a6d0-47f6-9187-5b68af6354e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:33.978082 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd8330-a6d0-47f6-9187-5b68af6354e8 admin admin] PATH_INFO: `/v3/users/191a771b2f214de991bec219d974fbfb` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:33.979177 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dd8330-a6d0-47f6-9187-5b68af6354e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=191a771b2f214de991bec219d974fbfb)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:33.981199 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dd8330-a6d0-47f6-9187-5b68af6354e8 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:34.187350 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-27dd8330-a6d0-47f6-9187-5b68af6354e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '191a771b2f214de991bec219d974fbfb'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:34.200645 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-27dd8330-a6d0-47f6-9187-5b68af6354e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '191a771b2f214de991bec219d974fbfb'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:34.201560 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-27dd8330-a6d0-47f6-9187-5b68af6354e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '191a771b2f214de991bec219d974fbfb'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:34.208961 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-27dd8330-a6d0-47f6-9187-5b68af6354e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '191a771b2f214de991bec219d974fbfb'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:34.289999 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-27dd8330-a6d0-47f6-9187-5b68af6354e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '191a771b2f214de991bec219d974fbfb'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:34.296424 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1271/2560] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:55:33 2022] DELETE /identity/v3/users/191a771b2f214de991bec219d974fbfb => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:55:34.820104 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431cfb42-ff84-4dd4-80c7-43eae86d46e9 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:34.821530 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-431cfb42-ff84-4dd4-80c7-43eae86d46e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:34.835833 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431cfb42-ff84-4dd4-80c7-43eae86d46e9 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:34.892368 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8454a9-71df-4006-88cf-7d6afdacb585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:34.892999 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8454a9-71df-4006-88cf-7d6afdacb585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:34.893274 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8454a9-71df-4006-88cf-7d6afdacb585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:34.893541 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8454a9-71df-4006-88cf-7d6afdacb585 admin admin] PATH_INFO: `/v3/projects/e4d478efd46a45f3827613c13b9e2480` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:34.894632 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8454a9-71df-4006-88cf-7d6afdacb585 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4d478efd46a45f3827613c13b9e2480)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:34.897074 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8454a9-71df-4006-88cf-7d6afdacb585 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:35.007701 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-fd8454a9-71df-4006-88cf-7d6afdacb585 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4d478efd46a45f3827613c13b9e2480', 'request_id': 'req-fd8454a9-71df-4006-88cf-7d6afdacb585'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:35.014776 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-fd8454a9-71df-4006-88cf-7d6afdacb585 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4d478efd46a45f3827613c13b9e2480', 'request_id': 'req-fd8454a9-71df-4006-88cf-7d6afdacb585'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:35.069890 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-fd8454a9-71df-4006-88cf-7d6afdacb585 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4d478efd46a45f3827613c13b9e2480', 'request_id': 'req-fd8454a9-71df-4006-88cf-7d6afdacb585'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:35.082788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-fd8454a9-71df-4006-88cf-7d6afdacb585 admin admin] The token cache is being invalidate because project e4d478efd46a45f3827613c13b9e2480 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:55:35.083080 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-fd8454a9-71df-4006-88cf-7d6afdacb585 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:35.084421 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-fd8454a9-71df-4006-88cf-7d6afdacb585 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:35.086014 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1290/2561] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:55:34 2022] DELETE /identity/v3/projects/e4d478efd46a45f3827613c13b9e2480 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:55:35.091607 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f53821-8ecc-4af0-a8e6-1603f6f1f1d6 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:35.092134 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f53821-8ecc-4af0-a8e6-1603f6f1f1d6 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:35.092134 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f53821-8ecc-4af0-a8e6-1603f6f1f1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:35.111765 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-56f53821-8ecc-4af0-a8e6-1603f6f1f1d6 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:55:35.112968 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-56f53821-8ecc-4af0-a8e6-1603f6f1f1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:35.156254 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:55:35.188721 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1272/2562] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:55:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:55:35.197339 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55719943-4bdc-4596-8190-a11fb1791343 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:35.197632 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55719943-4bdc-4596-8190-a11fb1791343 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:35.197964 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55719943-4bdc-4596-8190-a11fb1791343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:35.220710 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-55719943-4bdc-4596-8190-a11fb1791343 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:55:35.221927 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-55719943-4bdc-4596-8190-a11fb1791343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:35.230598 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1291/2563] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:55:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:55:35.238574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f53821-8ecc-4af0-a8e6-1603f6f1f1d6 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:35.247897 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f53821-8ecc-4af0-a8e6-1603f6f1f1d6 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:35.254357 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc7824-390d-48ae-8309-df3d76f3c4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:35.255553 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc7824-390d-48ae-8309-df3d76f3c4a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:35.256094 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc7824-390d-48ae-8309-df3d76f3c4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:35.256436 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc7824-390d-48ae-8309-df3d76f3c4a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:35.257091 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bc7824-390d-48ae-8309-df3d76f3c4a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:55:35.257397 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bc7824-390d-48ae-8309-df3d76f3c4a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:35.261237 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bc7824-390d-48ae-8309-df3d76f3c4a4 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:35.276375 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1273/2564] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:55:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:55:35.284789 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55719943-4bdc-4596-8190-a11fb1791343 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:35.295704 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55719943-4bdc-4596-8190-a11fb1791343 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:35.300971 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f64db2-75db-49a1-8e01-9f4320f541da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:35.301240 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f64db2-75db-49a1-8e01-9f4320f541da admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:35.301526 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f64db2-75db-49a1-8e01-9f4320f541da admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:35.301797 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f64db2-75db-49a1-8e01-9f4320f541da admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:35.302629 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f64db2-75db-49a1-8e01-9f4320f541da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:35.304972 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f64db2-75db-49a1-8e01-9f4320f541da admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:35.310255 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:55:35.473571 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1292/2565] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:55:35 2022] POST /identity/v3/projects => generated 383 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:55:35.477876 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc7824-390d-48ae-8309-df3d76f3c4a4 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:35.515390 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc7824-390d-48ae-8309-df3d76f3c4a4 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:35.584650 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e9f68-7ad3-4655-b1b6-26ab4fe65565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:35.585279 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e9f68-7ad3-4655-b1b6-26ab4fe65565 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:35.585610 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e9f68-7ad3-4655-b1b6-26ab4fe65565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:35.585896 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e9f68-7ad3-4655-b1b6-26ab4fe65565 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:35.586693 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7e9f68-7ad3-4655-b1b6-26ab4fe65565 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:35.589787 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7e9f68-7ad3-4655-b1b6-26ab4fe65565 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:35.725750 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1274/2566] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:55:35 2022] POST /identity/v3/users => generated 292 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:55:35.732065 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f64db2-75db-49a1-8e01-9f4320f541da admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:35.746507 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f64db2-75db-49a1-8e01-9f4320f541da admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:35.751874 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f258cf-8885-4d7f-9ea8-a386865bf8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:35.752502 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f258cf-8885-4d7f-9ea8-a386865bf8d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:35.752805 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f258cf-8885-4d7f-9ea8-a386865bf8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:35.753058 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f258cf-8885-4d7f-9ea8-a386865bf8d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:35.754189 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f258cf-8885-4d7f-9ea8-a386865bf8d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:35.756599 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f258cf-8885-4d7f-9ea8-a386865bf8d1 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:35.772062 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1293/2567] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:55:35.775201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e9f68-7ad3-4655-b1b6-26ab4fe65565 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:35.788992 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e9f68-7ad3-4655-b1b6-26ab4fe65565 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:35.793922 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd56fd-d98c-406b-b162-e95388ad2239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:35.794529 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd56fd-d98c-406b-b162-e95388ad2239 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:35.794857 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd56fd-d98c-406b-b162-e95388ad2239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:35.795078 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd56fd-d98c-406b-b162-e95388ad2239 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:35.795733 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fd56fd-d98c-406b-b162-e95388ad2239 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:35.797348 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fd56fd-d98c-406b-b162-e95388ad2239 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:35.809324 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1275/2568] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:55:35.813769 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f258cf-8885-4d7f-9ea8-a386865bf8d1 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:35.826071 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f258cf-8885-4d7f-9ea8-a386865bf8d1 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:35.832972 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b545d5-3122-49a4-b248-93c3f579f406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:35.834252 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b545d5-3122-49a4-b248-93c3f579f406 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:35.834670 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b545d5-3122-49a4-b248-93c3f579f406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:35.835085 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b545d5-3122-49a4-b248-93c3f579f406 admin admin] PATH_INFO: `/v3/projects/74de5aa72b134e2daf2f5ee0350d97a8/users/9e3ede4f82334a27922bc7b390adde0f/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:35.857517 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b545d5-3122-49a4-b248-93c3f579f406 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74de5aa72b134e2daf2f5ee0350d97a8, user_id=9e3ede4f82334a27922bc7b390adde0f, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:35.860531 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b545d5-3122-49a4-b248-93c3f579f406 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:35.910234 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1294/2569] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:55:35 2022] PUT /identity/v3/projects/74de5aa72b134e2daf2f5ee0350d97a8/users/9e3ede4f82334a27922bc7b390adde0f/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:55:35.921089 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2510c4a-6b3d-466a-b1f9-0c89bafa954e None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:35.921372 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2510c4a-6b3d-466a-b1f9-0c89bafa954e None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:35.921861 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2510c4a-6b3d-466a-b1f9-0c89bafa954e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:35.955476 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-b2510c4a-6b3d-466a-b1f9-0c89bafa954e None None] MFA Rules not processed for user `9e3ede4f82334a27922bc7b390adde0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:55:35.957022 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b2510c4a-6b3d-466a-b1f9-0c89bafa954e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:36.030313 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1276/2570] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:55:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:55:36.042582 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b545d5-3122-49a4-b248-93c3f579f406 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:36.045117 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-17b545d5-3122-49a4-b248-93c3f579f406 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:36.111064 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b545d5-3122-49a4-b248-93c3f579f406 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:36.115147 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faad7b2-d5fa-4e1a-8cbf-f95bb007c992 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:36.115437 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faad7b2-d5fa-4e1a-8cbf-f95bb007c992 service cinder] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:36.115719 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faad7b2-d5fa-4e1a-8cbf-f95bb007c992 service cinder] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:36.116278 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faad7b2-d5fa-4e1a-8cbf-f95bb007c992 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:36.125402 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faad7b2-d5fa-4e1a-8cbf-f95bb007c992 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:36.126808 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faad7b2-d5fa-4e1a-8cbf-f95bb007c992 service cinder] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:36.138563 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1295/2571] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:55:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:55:36.283923 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2510c4a-6b3d-466a-b1f9-0c89bafa954e None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:36.285302 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b2510c4a-6b3d-466a-b1f9-0c89bafa954e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:36.339996 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:55:36.352136 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2510c4a-6b3d-466a-b1f9-0c89bafa954e None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:36.355689 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b890a11c-8b67-4010-bf7d-439aad1f5124 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:36.356295 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b890a11c-8b67-4010-bf7d-439aad1f5124 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:36.356572 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b890a11c-8b67-4010-bf7d-439aad1f5124 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:36.356842 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b890a11c-8b67-4010-bf7d-439aad1f5124 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:36.358591 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b890a11c-8b67-4010-bf7d-439aad1f5124 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:36.369479 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b890a11c-8b67-4010-bf7d-439aad1f5124 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:36.370896 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b890a11c-8b67-4010-bf7d-439aad1f5124 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:36.378947 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1277/2572] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:55:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:55:38.555451 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020ee71-b8d6-43d2-ab5b-e839230acb3e None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:38.555917 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020ee71-b8d6-43d2-ab5b-e839230acb3e None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:38.556010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020ee71-b8d6-43d2-ab5b-e839230acb3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:38.576193 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-8020ee71-b8d6-43d2-ab5b-e839230acb3e None None] MFA Rules not processed for user `2e6e4387a7c24bbc82c22eefe2a3d138`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:55:38.577409 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-8020ee71-b8d6-43d2-ab5b-e839230acb3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:38.651709 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1296/2573] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Oct 3 12:55:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:55:38.847968 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b890a11c-8b67-4010-bf7d-439aad1f5124 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:38.852110 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b890a11c-8b67-4010-bf7d-439aad1f5124 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:38.940726 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b890a11c-8b67-4010-bf7d-439aad1f5124 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:38.955047 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1af2c3-e06c-4767-8c4a-a2606657b063 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b111d72a34144073917de4f566a58233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:38.955047 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1af2c3-e06c-4767-8c4a-a2606657b063 service swift] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:38.955047 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1af2c3-e06c-4767-8c4a-a2606657b063 service swift] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:38.955047 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1af2c3-e06c-4767-8c4a-a2606657b063 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:38.960465 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1af2c3-e06c-4767-8c4a-a2606657b063 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:38.962068 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1af2c3-e06c-4767-8c4a-a2606657b063 service swift] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:38.970241 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1278/2574] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Oct 3 12:55:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 12:55:39.463653 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020ee71-b8d6-43d2-ab5b-e839230acb3e None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:39.583946 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:55:39.590402 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020ee71-b8d6-43d2-ab5b-e839230acb3e None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:39.596240 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b3c82-caed-4f02-a11b-a838bffbe41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:39.597098 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b3c82-caed-4f02-a11b-a838bffbe41d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:39.597549 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b3c82-caed-4f02-a11b-a838bffbe41d admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:39.598002 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b3c82-caed-4f02-a11b-a838bffbe41d admin admin] PATH_INFO: `/v3/users/9e3ede4f82334a27922bc7b390adde0f` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:39.615082 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8b3c82-caed-4f02-a11b-a838bffbe41d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e3ede4f82334a27922bc7b390adde0f)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:39.629112 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8b3c82-caed-4f02-a11b-a838bffbe41d admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:39.658941 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1af2c3-e06c-4767-8c4a-a2606657b063 service swift] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:39.660750 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ad1af2c3-e06c-4767-8c4a-a2606657b063 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:39.775421 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1af2c3-e06c-4767-8c4a-a2606657b063 service swift] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:39.781659 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a105c4c8-a3a5-467c-a798-70ea7e6b57ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:39.782778 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105c4c8-a3a5-467c-a798-70ea7e6b57ea service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:39.783128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105c4c8-a3a5-467c-a798-70ea7e6b57ea service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:39.783606 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105c4c8-a3a5-467c-a798-70ea7e6b57ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:39.792594 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a105c4c8-a3a5-467c-a798-70ea7e6b57ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:39.808161 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a105c4c8-a3a5-467c-a798-70ea7e6b57ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:39.812663 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a105c4c8-a3a5-467c-a798-70ea7e6b57ea service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:39.827217 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1279/2575] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:55:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:55:39.897091 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-bd8b3c82-caed-4f02-a11b-a838bffbe41d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e3ede4f82334a27922bc7b390adde0f'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:39.904760 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-bd8b3c82-caed-4f02-a11b-a838bffbe41d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e3ede4f82334a27922bc7b390adde0f'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:39.920550 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-bd8b3c82-caed-4f02-a11b-a838bffbe41d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e3ede4f82334a27922bc7b390adde0f'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:39.921301 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-bd8b3c82-caed-4f02-a11b-a838bffbe41d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e3ede4f82334a27922bc7b390adde0f'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:39.995473 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-bd8b3c82-caed-4f02-a11b-a838bffbe41d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e3ede4f82334a27922bc7b390adde0f'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:40.004599 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1297/2576] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:55:39 2022] DELETE /identity/v3/users/9e3ede4f82334a27922bc7b390adde0f => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:55:40.009627 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5f5c8-cb85-4c18-8062-0ddc1e997743 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:40.015572 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5f5c8-cb85-4c18-8062-0ddc1e997743 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:40.015572 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5f5c8-cb85-4c18-8062-0ddc1e997743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:40.030699 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-f5f5f5c8-cb85-4c18-8062-0ddc1e997743 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:55:40.032261 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-f5f5f5c8-cb85-4c18-8062-0ddc1e997743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:40.087782 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:55:40.096839 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1280/2577] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:55:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:55:40.107591 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b3c82-caed-4f02-a11b-a838bffbe41d admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:40.109176 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-bd8b3c82-caed-4f02-a11b-a838bffbe41d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:40.171919 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b3c82-caed-4f02-a11b-a838bffbe41d admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:40.176954 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f240fbf-6efe-4850-9cb4-4050f2cd8982 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:40.177636 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f240fbf-6efe-4850-9cb4-4050f2cd8982 service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:40.177932 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f240fbf-6efe-4850-9cb4-4050f2cd8982 service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:40.178213 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f240fbf-6efe-4850-9cb4-4050f2cd8982 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:40.187787 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f240fbf-6efe-4850-9cb4-4050f2cd8982 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:40.189270 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f240fbf-6efe-4850-9cb4-4050f2cd8982 service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:40.199353 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1298/2578] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:55:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:55:40.629499 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f5f5c8-cb85-4c18-8062-0ddc1e997743 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:40.631296 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-f5f5f5c8-cb85-4c18-8062-0ddc1e997743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:40.667043 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f5f5c8-cb85-4c18-8062-0ddc1e997743 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:40.672931 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1686d-11df-4b9a-8ef6-1f46171ee9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:40.673589 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1686d-11df-4b9a-8ef6-1f46171ee9a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:40.675290 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1686d-11df-4b9a-8ef6-1f46171ee9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:40.675290 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1686d-11df-4b9a-8ef6-1f46171ee9a7 admin admin] PATH_INFO: `/v3/projects/74de5aa72b134e2daf2f5ee0350d97a8` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:40.675290 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f1686d-11df-4b9a-8ef6-1f46171ee9a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74de5aa72b134e2daf2f5ee0350d97a8)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:40.678102 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f1686d-11df-4b9a-8ef6-1f46171ee9a7 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:40.803745 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-05f1686d-11df-4b9a-8ef6-1f46171ee9a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74de5aa72b134e2daf2f5ee0350d97a8', 'request_id': 'req-05f1686d-11df-4b9a-8ef6-1f46171ee9a7'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:40.814217 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-05f1686d-11df-4b9a-8ef6-1f46171ee9a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74de5aa72b134e2daf2f5ee0350d97a8', 'request_id': 'req-05f1686d-11df-4b9a-8ef6-1f46171ee9a7'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:40.907856 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-05f1686d-11df-4b9a-8ef6-1f46171ee9a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74de5aa72b134e2daf2f5ee0350d97a8', 'request_id': 'req-05f1686d-11df-4b9a-8ef6-1f46171ee9a7'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:40.936219 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-05f1686d-11df-4b9a-8ef6-1f46171ee9a7 admin admin] The token cache is being invalidate because project 74de5aa72b134e2daf2f5ee0350d97a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:55:40.936575 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-05f1686d-11df-4b9a-8ef6-1f46171ee9a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:40.940747 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-05f1686d-11df-4b9a-8ef6-1f46171ee9a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:55:40.949640 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1281/2579] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:55:40 2022] DELETE /identity/v3/projects/74de5aa72b134e2daf2f5ee0350d97a8 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:55:40.959766 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c28f4-bc6f-43b7-a055-c09212bccf2f None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:40.960599 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c28f4-bc6f-43b7-a055-c09212bccf2f None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:40.961030 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c28f4-bc6f-43b7-a055-c09212bccf2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:41.009001 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-ce9c28f4-bc6f-43b7-a055-c09212bccf2f None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:55:41.023957 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-ce9c28f4-bc6f-43b7-a055-c09212bccf2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:41.097746 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1299/2580] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:55:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:55:41.105318 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec78879-c3a2-4eac-94d0-17641acf594c None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:41.105817 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec78879-c3a2-4eac-94d0-17641acf594c None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:41.105936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec78879-c3a2-4eac-94d0-17641acf594c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:41.134149 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-1ec78879-c3a2-4eac-94d0-17641acf594c None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:55:41.135435 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1ec78879-c3a2-4eac-94d0-17641acf594c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:41.145473 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1282/2581] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:55:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:55:41.150308 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c28f4-bc6f-43b7-a055-c09212bccf2f None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:41.163748 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c28f4-bc6f-43b7-a055-c09212bccf2f None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:41.169008 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb52522-a2a6-4ca7-a978-0076389b264b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:41.169728 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb52522-a2a6-4ca7-a978-0076389b264b admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:41.170069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb52522-a2a6-4ca7-a978-0076389b264b admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:41.170467 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb52522-a2a6-4ca7-a978-0076389b264b admin admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:41.171174 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb52522-a2a6-4ca7-a978-0076389b264b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:55:41.171572 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb52522-a2a6-4ca7-a978-0076389b264b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:41.173233 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb52522-a2a6-4ca7-a978-0076389b264b admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:41.191357 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1300/2582] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:55:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:55:41.196062 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec78879-c3a2-4eac-94d0-17641acf594c None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:41.206463 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec78879-c3a2-4eac-94d0-17641acf594c None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:41.212282 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e1d49-257c-4b1e-851c-849dbe69e532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:41.212897 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e1d49-257c-4b1e-851c-849dbe69e532 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:41.213277 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e1d49-257c-4b1e-851c-849dbe69e532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:41.213557 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e1d49-257c-4b1e-851c-849dbe69e532 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:41.214187 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e1d49-257c-4b1e-851c-849dbe69e532 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:41.216875 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e1d49-257c-4b1e-851c-849dbe69e532 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:41.325332 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1283/2583] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:55:41 2022] POST /identity/v3/projects => generated 375 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:55:41.326855 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb52522-a2a6-4ca7-a978-0076389b264b admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:41.338761 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb52522-a2a6-4ca7-a978-0076389b264b admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:41.342726 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:55:41.409942 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e8150c-1b21-496c-845c-2751f838d09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:41.410582 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e8150c-1b21-496c-845c-2751f838d09f admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:41.410853 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e8150c-1b21-496c-845c-2751f838d09f admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:41.411843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e8150c-1b21-496c-845c-2751f838d09f admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:41.413009 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e8150c-1b21-496c-845c-2751f838d09f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:41.415711 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e8150c-1b21-496c-845c-2751f838d09f admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:41.522132 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1301/2584] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:55:41 2022] POST /identity/v3/users => generated 288 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:55:41.526548 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e1d49-257c-4b1e-851c-849dbe69e532 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:41.540572 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e1d49-257c-4b1e-851c-849dbe69e532 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:41.545303 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a5489-65f9-44c0-bc8e-6c01fa6e9053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:41.546014 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a5489-65f9-44c0-bc8e-6c01fa6e9053 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:41.546382 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a5489-65f9-44c0-bc8e-6c01fa6e9053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:41.546841 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a5489-65f9-44c0-bc8e-6c01fa6e9053 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:41.548615 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46a5489-65f9-44c0-bc8e-6c01fa6e9053 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:41.550850 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46a5489-65f9-44c0-bc8e-6c01fa6e9053 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:41.563919 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1284/2585] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:55:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:55:41.567997 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e8150c-1b21-496c-845c-2751f838d09f admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:41.580424 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e8150c-1b21-496c-845c-2751f838d09f admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:41.585282 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554c0b5-6ee7-4f15-ad0c-d7783efb6081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:41.586070 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554c0b5-6ee7-4f15-ad0c-d7783efb6081 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:41.586424 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554c0b5-6ee7-4f15-ad0c-d7783efb6081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:41.587918 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554c0b5-6ee7-4f15-ad0c-d7783efb6081 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:41.587918 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d554c0b5-6ee7-4f15-ad0c-d7783efb6081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:41.590148 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d554c0b5-6ee7-4f15-ad0c-d7783efb6081 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:41.604915 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1302/2586] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:55:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 12:55:41.606614 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a5489-65f9-44c0-bc8e-6c01fa6e9053 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:41.618502 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a5489-65f9-44c0-bc8e-6c01fa6e9053 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:41.623246 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d6de5f-a4fd-48ec-afaa-f5b59e00bacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:41.623933 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d6de5f-a4fd-48ec-afaa-f5b59e00bacc admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:41.624210 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d6de5f-a4fd-48ec-afaa-f5b59e00bacc admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:41.624488 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d6de5f-a4fd-48ec-afaa-f5b59e00bacc admin admin] PATH_INFO: `/v3/projects/ead596751f234d79b38d661b0d5c6343/users/d3b8ec0638764aba88e1920ecca7ddbc/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:41.644357 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d6de5f-a4fd-48ec-afaa-f5b59e00bacc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ead596751f234d79b38d661b0d5c6343, user_id=d3b8ec0638764aba88e1920ecca7ddbc, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:41.647174 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d6de5f-a4fd-48ec-afaa-f5b59e00bacc admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:41.717856 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1285/2587] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:55:41 2022] PUT /identity/v3/projects/ead596751f234d79b38d661b0d5c6343/users/d3b8ec0638764aba88e1920ecca7ddbc/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:55:41.787197 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690edaae-5c04-446e-93e9-a363f256bffe None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:41.787600 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690edaae-5c04-446e-93e9-a363f256bffe None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:41.787881 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690edaae-5c04-446e-93e9-a363f256bffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:41.819794 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-690edaae-5c04-446e-93e9-a363f256bffe None None] MFA Rules not processed for user `d3b8ec0638764aba88e1920ecca7ddbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:55:41.820939 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-690edaae-5c04-446e-93e9-a363f256bffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:41.898116 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1303/2588] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:55:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:55:41.908952 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d6de5f-a4fd-48ec-afaa-f5b59e00bacc admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:41.910977 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b3d6de5f-a4fd-48ec-afaa-f5b59e00bacc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:41.980487 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d6de5f-a4fd-48ec-afaa-f5b59e00bacc admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:41.984348 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c8e07-739b-489b-af92-114c0124304d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:41.985093 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c8e07-739b-489b-af92-114c0124304d service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:41.985370 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c8e07-739b-489b-af92-114c0124304d service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:41.985656 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c8e07-739b-489b-af92-114c0124304d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:41.995554 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124c8e07-739b-489b-af92-114c0124304d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:41.997056 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124c8e07-739b-489b-af92-114c0124304d service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:42.008068 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1286/2589] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:55:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:55:44.189402 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b779a620-075f-4874-b567-83f9e4b8ef16 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:44.189956 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b779a620-075f-4874-b567-83f9e4b8ef16 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:44.189956 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b779a620-075f-4874-b567-83f9e4b8ef16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:44.211909 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-b779a620-075f-4874-b567-83f9e4b8ef16 None None] MFA Rules not processed for user `d3b8ec0638764aba88e1920ecca7ddbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:55:44.213160 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-b779a620-075f-4874-b567-83f9e4b8ef16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:44.220949 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1304/2590] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:55:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:55:51.543527 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c8e07-739b-489b-af92-114c0124304d service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:55:51.546536 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-124c8e07-739b-489b-af92-114c0124304d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:51.592763 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:55:51.666724 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c8e07-739b-489b-af92-114c0124304d service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:55:51.672305 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ad1aa-90e1-4c69-b682-c2e9825b98ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:55:51.672950 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ad1aa-90e1-4c69-b682-c2e9825b98ac service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:55:51.673286 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ad1aa-90e1-4c69-b682-c2e9825b98ac service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:55:51.674074 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ad1aa-90e1-4c69-b682-c2e9825b98ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:55:51.675904 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-9b5ad1aa-90e1-4c69-b682-c2e9825b98ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:55:51.690608 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5ad1aa-90e1-4c69-b682-c2e9825b98ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:55:51.692737 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5ad1aa-90e1-4c69-b682-c2e9825b98ac service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:55:51.719202 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1287/2591] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:55:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:55:54.206002 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=66,keystone:INSERT=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:56:02.044398 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=24 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:56:04.182557 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b2c34-d293-40da-b36f-895bb1323d12 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:56:04.183120 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b2c34-d293-40da-b36f-895bb1323d12 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:56:04.183120 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b2c34-d293-40da-b36f-895bb1323d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:56:04.203112 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-7a6b2c34-d293-40da-b36f-895bb1323d12 None None] MFA Rules not processed for user `2e6e4387a7c24bbc82c22eefe2a3d138`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:56:04.204411 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-7a6b2c34-d293-40da-b36f-895bb1323d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:56:04.259444 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1305/2592] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Oct 3 12:56:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:56:04.272691 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ad1aa-90e1-4c69-b682-c2e9825b98ac service nova] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:56:04.274425 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-9b5ad1aa-90e1-4c69-b682-c2e9825b98ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:56:04.344991 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ad1aa-90e1-4c69-b682-c2e9825b98ac service nova] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:56:04.350356 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41801a03-870d-4a1a-9460-2193c6b240bd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b111d72a34144073917de4f566a58233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:56:04.350966 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41801a03-870d-4a1a-9460-2193c6b240bd service swift] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:56:04.351245 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41801a03-870d-4a1a-9460-2193c6b240bd service swift] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:56:04.351551 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41801a03-870d-4a1a-9460-2193c6b240bd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:56:04.359530 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41801a03-870d-4a1a-9460-2193c6b240bd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:56:04.362034 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41801a03-870d-4a1a-9460-2193c6b240bd service swift] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:56:04.370574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1288/2593] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Oct 3 12:56:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 12:56:05.611584 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9029521-7d6f-48fc-ad6d-2848b102f5af None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:56:05.612157 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9029521-7d6f-48fc-ad6d-2848b102f5af None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:56:05.612157 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9029521-7d6f-48fc-ad6d-2848b102f5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:56:05.628429 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41801a03-870d-4a1a-9460-2193c6b240bd service swift] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:56:05.630633 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-41801a03-870d-4a1a-9460-2193c6b240bd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:56:05.645884 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-e9029521-7d6f-48fc-ad6d-2848b102f5af None None] MFA Rules not processed for user `2e6e4387a7c24bbc82c22eefe2a3d138`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:56:05.647097 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-e9029521-7d6f-48fc-ad6d-2848b102f5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:56:05.658187 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1306/2594] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Oct 3 12:56:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:56:05.684084 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9029521-7d6f-48fc-ad6d-2848b102f5af None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:56:05.698114 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9029521-7d6f-48fc-ad6d-2848b102f5af None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:56:05.705555 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9eba92-7625-4cf3-91b6-f0f9b6a731e2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b111d72a34144073917de4f566a58233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:56:05.711304 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9eba92-7625-4cf3-91b6-f0f9b6a731e2 service swift] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:56:05.711304 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9eba92-7625-4cf3-91b6-f0f9b6a731e2 service swift] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:56:05.711304 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9eba92-7625-4cf3-91b6-f0f9b6a731e2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:56:05.712628 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41801a03-870d-4a1a-9460-2193c6b240bd service swift] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:56:05.716432 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f9abd-a12f-4d19-8e27-3f1cfefe8edb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:56:05.717383 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f9abd-a12f-4d19-8e27-3f1cfefe8edb service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:56:05.717807 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f9abd-a12f-4d19-8e27-3f1cfefe8edb service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:56:05.718239 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f9abd-a12f-4d19-8e27-3f1cfefe8edb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:56:05.720188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-120f9abd-a12f-4d19-8e27-3f1cfefe8edb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:56:05.723223 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9eba92-7625-4cf3-91b6-f0f9b6a731e2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:56:05.725146 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9eba92-7625-4cf3-91b6-f0f9b6a731e2 service swift] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:56:05.737300 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1307/2595] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Oct 3 12:56:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 12:56:05.804312 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f9abd-a12f-4d19-8e27-3f1cfefe8edb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:56:05.805864 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f9abd-a12f-4d19-8e27-3f1cfefe8edb service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:56:05.817247 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1289/2596] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:56:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:56:06.933407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7bae38-ff03-44ca-8f99-e9c52498c1da None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:56:06.934280 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7bae38-ff03-44ca-8f99-e9c52498c1da None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:56:06.934692 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7bae38-ff03-44ca-8f99-e9c52498c1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:56:06.956477 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-6d7bae38-ff03-44ca-8f99-e9c52498c1da None None] MFA Rules not processed for user `2e6e4387a7c24bbc82c22eefe2a3d138`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:56:06.958624 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-6d7bae38-ff03-44ca-8f99-e9c52498c1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:56:06.974629 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1308/2597] 10.222.0.33 () {58 vars in 967 bytes} [Mon Oct 3 12:56:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:56:06.985162 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f9abd-a12f-4d19-8e27-3f1cfefe8edb service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:56:07.001013 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f9abd-a12f-4d19-8e27-3f1cfefe8edb service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:56:07.007205 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9664fc20-5d59-4dd2-9f3c-83810c0c7ac6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b111d72a34144073917de4f566a58233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:56:07.007956 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9664fc20-5d59-4dd2-9f3c-83810c0c7ac6 service swift] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:56:07.008323 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9664fc20-5d59-4dd2-9f3c-83810c0c7ac6 service swift] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:56:07.008745 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9664fc20-5d59-4dd2-9f3c-83810c0c7ac6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:56:07.022147 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9664fc20-5d59-4dd2-9f3c-83810c0c7ac6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:56:07.023985 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9664fc20-5d59-4dd2-9f3c-83810c0c7ac6 service swift] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:56:07.033325 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1290/2598] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Oct 3 12:56:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 12:56:08.209688 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c507e-a390-4354-9415-6c77aeecc8f1 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:56:08.210848 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c507e-a390-4354-9415-6c77aeecc8f1 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:56:08.211402 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c507e-a390-4354-9415-6c77aeecc8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:56:08.262407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-951c507e-a390-4354-9415-6c77aeecc8f1 None None] MFA Rules not processed for user `2e6e4387a7c24bbc82c22eefe2a3d138`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:56:08.264080 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-951c507e-a390-4354-9415-6c77aeecc8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:56:08.289863 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1309/2599] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Oct 3 12:56:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:56:08.315891 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9664fc20-5d59-4dd2-9f3c-83810c0c7ac6 service swift] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:56:08.334490 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9664fc20-5d59-4dd2-9f3c-83810c0c7ac6 service swift] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:56:08.345308 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab34db-9d46-4df1-b003-f42a91903684 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b111d72a34144073917de4f566a58233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:56:08.346298 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab34db-9d46-4df1-b003-f42a91903684 service swift] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:56:08.346770 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab34db-9d46-4df1-b003-f42a91903684 service swift] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:56:08.347396 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab34db-9d46-4df1-b003-f42a91903684 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:56:08.361963 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ab34db-9d46-4df1-b003-f42a91903684 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:56:08.373776 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ab34db-9d46-4df1-b003-f42a91903684 service swift] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:56:08.401239 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1291/2600] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Oct 3 12:56:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 12:56:09.716848 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320b846-8856-461c-88d1-d032834b2548 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:56:09.718012 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320b846-8856-461c-88d1-d032834b2548 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:56:09.718458 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320b846-8856-461c-88d1-d032834b2548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:56:09.744164 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-5320b846-8856-461c-88d1-d032834b2548 None None] MFA Rules not processed for user `2e6e4387a7c24bbc82c22eefe2a3d138`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:56:09.745670 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-5320b846-8856-461c-88d1-d032834b2548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:56:09.757998 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1310/2601] 10.222.0.33 () {58 vars in 967 bytes} [Mon Oct 3 12:56:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:56:09.789369 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab34db-9d46-4df1-b003-f42a91903684 service swift] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:56:09.808825 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab34db-9d46-4df1-b003-f42a91903684 service swift] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:56:09.820179 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcad968-7d50-4d8a-b6a3-71160cce626a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b111d72a34144073917de4f566a58233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:56:09.820828 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcad968-7d50-4d8a-b6a3-71160cce626a service swift] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:56:09.821111 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcad968-7d50-4d8a-b6a3-71160cce626a service swift] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:56:09.821387 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcad968-7d50-4d8a-b6a3-71160cce626a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:56:09.851511 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcad968-7d50-4d8a-b6a3-71160cce626a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:56:09.853369 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcad968-7d50-4d8a-b6a3-71160cce626a service swift] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:56:09.870478 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=100 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:56:09.876227 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1292/2602] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Oct 3 12:56:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 12:56:12.437242 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320b846-8856-461c-88d1-d032834b2548 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:56:12.437242 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-5320b846-8856-461c-88d1-d032834b2548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:56:12.454224 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320b846-8856-461c-88d1-d032834b2548 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:56:12.464017 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710fad28-9349-4cdd-890f-0b33bb319ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:56:12.464569 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710fad28-9349-4cdd-890f-0b33bb319ea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:56:12.464906 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710fad28-9349-4cdd-890f-0b33bb319ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:56:12.465108 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710fad28-9349-4cdd-890f-0b33bb319ea6 admin admin] PATH_INFO: `/v3/users/bd24d209dc9644e8b2860a0e676acd3b` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:56:12.466344 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710fad28-9349-4cdd-890f-0b33bb319ea6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd24d209dc9644e8b2860a0e676acd3b)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:56:12.468483 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710fad28-9349-4cdd-890f-0b33bb319ea6 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:56:12.663423 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-710fad28-9349-4cdd-890f-0b33bb319ea6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd24d209dc9644e8b2860a0e676acd3b'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:56:12.680403 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-710fad28-9349-4cdd-890f-0b33bb319ea6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd24d209dc9644e8b2860a0e676acd3b'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:56:12.712698 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-710fad28-9349-4cdd-890f-0b33bb319ea6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd24d209dc9644e8b2860a0e676acd3b'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:56:12.713441 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-710fad28-9349-4cdd-890f-0b33bb319ea6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd24d209dc9644e8b2860a0e676acd3b'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:56:12.809358 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-710fad28-9349-4cdd-890f-0b33bb319ea6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd24d209dc9644e8b2860a0e676acd3b'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:56:12.816068 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1311/2603] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:56:12 2022] DELETE /identity/v3/users/bd24d209dc9644e8b2860a0e676acd3b => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:56:13.167160 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcad968-7d50-4d8a-b6a3-71160cce626a service swift] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:56:13.179252 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcad968-7d50-4d8a-b6a3-71160cce626a service swift] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:56:13.250438 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1856ea-96ac-4d73-855b-fbe78a62e18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:56:13.257693 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1856ea-96ac-4d73-855b-fbe78a62e18c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:56:13.257693 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1856ea-96ac-4d73-855b-fbe78a62e18c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:56:13.257693 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1856ea-96ac-4d73-855b-fbe78a62e18c admin admin] PATH_INFO: `/v3/projects/cc40d256f0e24050b6b53fdbcbce1962` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:56:13.257693 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1856ea-96ac-4d73-855b-fbe78a62e18c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc40d256f0e24050b6b53fdbcbce1962)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:56:13.257693 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1856ea-96ac-4d73-855b-fbe78a62e18c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:56:13.379510 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9e1856ea-96ac-4d73-855b-fbe78a62e18c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc40d256f0e24050b6b53fdbcbce1962', 'request_id': 'req-9e1856ea-96ac-4d73-855b-fbe78a62e18c'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:56:13.385219 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9e1856ea-96ac-4d73-855b-fbe78a62e18c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc40d256f0e24050b6b53fdbcbce1962', 'request_id': 'req-9e1856ea-96ac-4d73-855b-fbe78a62e18c'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:56:13.461227 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9e1856ea-96ac-4d73-855b-fbe78a62e18c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc40d256f0e24050b6b53fdbcbce1962', 'request_id': 'req-9e1856ea-96ac-4d73-855b-fbe78a62e18c'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:56:13.476090 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9e1856ea-96ac-4d73-855b-fbe78a62e18c admin admin] The token cache is being invalidate because project cc40d256f0e24050b6b53fdbcbce1962 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:56:13.476484 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9e1856ea-96ac-4d73-855b-fbe78a62e18c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:56:13.477266 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-9e1856ea-96ac-4d73-855b-fbe78a62e18c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:56:13.478826 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1293/2604] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:56:13 2022] DELETE /identity/v3/projects/cc40d256f0e24050b6b53fdbcbce1962 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:56:13.483868 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba9a6e-effe-47c7-a661-42658e073870 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:56:13.484381 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba9a6e-effe-47c7-a661-42658e073870 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:56:13.484484 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba9a6e-effe-47c7-a661-42658e073870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:56:13.506925 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-e3ba9a6e-effe-47c7-a661-42658e073870 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:56:13.508323 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-e3ba9a6e-effe-47c7-a661-42658e073870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:56:13.543748 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:56:13.576532 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1312/2605] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:56:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:56:13.584600 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983abb45-7531-40b6-a77c-1c667b5d9b82 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:56:13.584888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983abb45-7531-40b6-a77c-1c667b5d9b82 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:56:13.585190 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983abb45-7531-40b6-a77c-1c667b5d9b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:56:13.605653 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-983abb45-7531-40b6-a77c-1c667b5d9b82 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:56:13.606863 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-983abb45-7531-40b6-a77c-1c667b5d9b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:56:13.615498 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1294/2606] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:56:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:56:13.619984 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba9a6e-effe-47c7-a661-42658e073870 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:56:13.629437 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba9a6e-effe-47c7-a661-42658e073870 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:56:13.634223 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ba23c-b984-47e3-a447-604113bda6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:56:13.634819 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ba23c-b984-47e3-a447-604113bda6b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:56:13.635098 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ba23c-b984-47e3-a447-604113bda6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:56:13.635454 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ba23c-b984-47e3-a447-604113bda6b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:56:13.636048 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368ba23c-b984-47e3-a447-604113bda6b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:56:13.636350 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368ba23c-b984-47e3-a447-604113bda6b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:56:13.637924 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368ba23c-b984-47e3-a447-604113bda6b9 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:56:13.649369 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1313/2607] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:56:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 12:56:13.653502 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983abb45-7531-40b6-a77c-1c667b5d9b82 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:56:13.663217 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983abb45-7531-40b6-a77c-1c667b5d9b82 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:56:13.667885 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62010105-230b-4671-9624-73832dcb679a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:56:13.668477 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62010105-230b-4671-9624-73832dcb679a admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:56:13.668757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62010105-230b-4671-9624-73832dcb679a admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:56:13.669029 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62010105-230b-4671-9624-73832dcb679a admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:56:13.669657 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62010105-230b-4671-9624-73832dcb679a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:56:13.671567 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62010105-230b-4671-9624-73832dcb679a admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:56:13.777398 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1295/2608] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:56:13 2022] POST /identity/v3/projects => generated 381 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:56:13.781460 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ba23c-b984-47e3-a447-604113bda6b9 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:56:13.792051 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ba23c-b984-47e3-a447-604113bda6b9 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:56:13.852539 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0c072-275c-44ee-87c6-255db1996a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:56:13.853180 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0c072-275c-44ee-87c6-255db1996a41 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:56:13.853463 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0c072-275c-44ee-87c6-255db1996a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:56:13.853776 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0c072-275c-44ee-87c6-255db1996a41 admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:56:13.854424 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af0c072-275c-44ee-87c6-255db1996a41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:56:13.856368 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af0c072-275c-44ee-87c6-255db1996a41 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:56:13.964467 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1314/2609] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:56:13 2022] POST /identity/v3/users => generated 291 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:56:13.968581 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62010105-230b-4671-9624-73832dcb679a admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:56:13.980134 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62010105-230b-4671-9624-73832dcb679a admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:56:13.984924 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd9dece-0bbe-4152-9626-098dcff904ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:56:13.985528 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9dece-0bbe-4152-9626-098dcff904ad admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:56:13.985809 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9dece-0bbe-4152-9626-098dcff904ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:56:13.986217 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9dece-0bbe-4152-9626-098dcff904ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:56:13.986746 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd9dece-0bbe-4152-9626-098dcff904ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:56:13.988465 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd9dece-0bbe-4152-9626-098dcff904ad admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:56:13.998496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1296/2610] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:56:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 12:56:14.002460 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0c072-275c-44ee-87c6-255db1996a41 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:56:14.012479 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0c072-275c-44ee-87c6-255db1996a41 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:56:14.017076 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c402c8-d2f9-4334-a2bc-69db66af3dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:56:14.017662 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c402c8-d2f9-4334-a2bc-69db66af3dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:56:14.017942 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c402c8-d2f9-4334-a2bc-69db66af3dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:56:14.018216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c402c8-d2f9-4334-a2bc-69db66af3dc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:56:14.018854 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c402c8-d2f9-4334-a2bc-69db66af3dc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:56:14.021229 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c402c8-d2f9-4334-a2bc-69db66af3dc1 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:56:14.031795 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1315/2611] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:56:14.036657 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd9dece-0bbe-4152-9626-098dcff904ad admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:56:14.046958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd9dece-0bbe-4152-9626-098dcff904ad admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:56:14.051607 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37e77f-1189-4409-89ef-da4727f92fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:56:14.052289 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37e77f-1189-4409-89ef-da4727f92fc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:56:14.052566 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37e77f-1189-4409-89ef-da4727f92fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:56:14.052865 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37e77f-1189-4409-89ef-da4727f92fc5 admin admin] PATH_INFO: `/v3/projects/484522301976426dae50e64c141c727e/users/5e29a4c0276247c286cff3eed5a1253a/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:56:14.068122 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d37e77f-1189-4409-89ef-da4727f92fc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=484522301976426dae50e64c141c727e, user_id=5e29a4c0276247c286cff3eed5a1253a, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:56:14.070735 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d37e77f-1189-4409-89ef-da4727f92fc5 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:56:14.160358 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1297/2612] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:56:14 2022] PUT /identity/v3/projects/484522301976426dae50e64c141c727e/users/5e29a4c0276247c286cff3eed5a1253a/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:56:14.165319 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5858c62-6ac1-4468-9113-48d9f328124c None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:56:14.165603 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5858c62-6ac1-4468-9113-48d9f328124c None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:56:14.165903 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5858c62-6ac1-4468-9113-48d9f328124c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:56:14.184497 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-e5858c62-6ac1-4468-9113-48d9f328124c None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:56:14.185695 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-e5858c62-6ac1-4468-9113-48d9f328124c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:56:14.243941 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1316/2613] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:56:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:56:14.253509 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37e77f-1189-4409-89ef-da4727f92fc5 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:56:14.255894 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-7d37e77f-1189-4409-89ef-da4727f92fc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:56:14.286664 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:56:14.312009 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37e77f-1189-4409-89ef-da4727f92fc5 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:56:14.316105 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff613b82-3a7c-4d2d-b1c7-ca7b5e43bcb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:56:14.317109 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff613b82-3a7c-4d2d-b1c7-ca7b5e43bcb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:56:14.317386 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff613b82-3a7c-4d2d-b1c7-ca7b5e43bcb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:56:14.317805 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff613b82-3a7c-4d2d-b1c7-ca7b5e43bcb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:56:14.329200 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff613b82-3a7c-4d2d-b1c7-ca7b5e43bcb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:56:14.330650 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff613b82-3a7c-4d2d-b1c7-ca7b5e43bcb1 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:56:14.339482 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5858c62-6ac1-4468-9113-48d9f328124c None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:56:14.340929 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-e5858c62-6ac1-4468-9113-48d9f328124c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:56:14.342010 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1298/2614] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:56:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:56:14.353157 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5858c62-6ac1-4468-9113-48d9f328124c None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:56:14.357746 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670249d5-4288-43b7-bea5-716d5fcd2c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:56:14.358374 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670249d5-4288-43b7-bea5-716d5fcd2c1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:56:14.358644 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670249d5-4288-43b7-bea5-716d5fcd2c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:56:14.358911 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670249d5-4288-43b7-bea5-716d5fcd2c1e admin admin] PATH_INFO: `/v3/users/d3b8ec0638764aba88e1920ecca7ddbc` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:56:14.360194 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670249d5-4288-43b7-bea5-716d5fcd2c1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3b8ec0638764aba88e1920ecca7ddbc)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:56:14.362212 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670249d5-4288-43b7-bea5-716d5fcd2c1e admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:56:14.656464 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:56:14.664807 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-670249d5-4288-43b7-bea5-716d5fcd2c1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3b8ec0638764aba88e1920ecca7ddbc'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:56:14.672576 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-670249d5-4288-43b7-bea5-716d5fcd2c1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3b8ec0638764aba88e1920ecca7ddbc'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:56:14.686416 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-670249d5-4288-43b7-bea5-716d5fcd2c1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3b8ec0638764aba88e1920ecca7ddbc'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:56:14.687506 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-670249d5-4288-43b7-bea5-716d5fcd2c1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3b8ec0638764aba88e1920ecca7ddbc'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:56:14.789242 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-670249d5-4288-43b7-bea5-716d5fcd2c1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3b8ec0638764aba88e1920ecca7ddbc'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:56:14.797325 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1317/2615] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:56:14 2022] DELETE /identity/v3/users/d3b8ec0638764aba88e1920ecca7ddbc => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:56:14.802518 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b612504-8c04-4806-9df9-2b298fc533d2 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:56:14.802805 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b612504-8c04-4806-9df9-2b298fc533d2 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:56:14.803081 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b612504-8c04-4806-9df9-2b298fc533d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:56:14.826682 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-1b612504-8c04-4806-9df9-2b298fc533d2 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:56:14.828182 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1b612504-8c04-4806-9df9-2b298fc533d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:56:14.914230 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1299/2616] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:56:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:56:14.923498 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670249d5-4288-43b7-bea5-716d5fcd2c1e admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:56:14.927484 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-670249d5-4288-43b7-bea5-716d5fcd2c1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:56:14.990762 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670249d5-4288-43b7-bea5-716d5fcd2c1e admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:56:14.994707 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee943a75-ed1d-4ac0-9c5d-0a2693aa06bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:56:14.995322 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee943a75-ed1d-4ac0-9c5d-0a2693aa06bc service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:56:14.995618 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee943a75-ed1d-4ac0-9c5d-0a2693aa06bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:56:14.995886 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee943a75-ed1d-4ac0-9c5d-0a2693aa06bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:56:15.005960 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee943a75-ed1d-4ac0-9c5d-0a2693aa06bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:56:15.007132 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee943a75-ed1d-4ac0-9c5d-0a2693aa06bc service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:56:15.020499 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1318/2617] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:56:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:56:15.367774 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b612504-8c04-4806-9df9-2b298fc533d2 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:56:15.369297 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1b612504-8c04-4806-9df9-2b298fc533d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:56:15.393255 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b612504-8c04-4806-9df9-2b298fc533d2 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:56:15.399395 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f2fc7-55ef-4895-be60-b6823c11a85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:56:15.400125 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f2fc7-55ef-4895-be60-b6823c11a85a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:56:15.400476 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f2fc7-55ef-4895-be60-b6823c11a85a admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:56:15.400822 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f2fc7-55ef-4895-be60-b6823c11a85a admin admin] PATH_INFO: `/v3/projects/ead596751f234d79b38d661b0d5c6343` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:56:15.402084 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f2fc7-55ef-4895-be60-b6823c11a85a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ead596751f234d79b38d661b0d5c6343)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:56:15.404225 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f2fc7-55ef-4895-be60-b6823c11a85a admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:56:15.551419 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-530f2fc7-55ef-4895-be60-b6823c11a85a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ead596751f234d79b38d661b0d5c6343', 'request_id': 'req-530f2fc7-55ef-4895-be60-b6823c11a85a'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:56:15.558065 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-530f2fc7-55ef-4895-be60-b6823c11a85a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ead596751f234d79b38d661b0d5c6343', 'request_id': 'req-530f2fc7-55ef-4895-be60-b6823c11a85a'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:56:15.622579 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-530f2fc7-55ef-4895-be60-b6823c11a85a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ead596751f234d79b38d661b0d5c6343', 'request_id': 'req-530f2fc7-55ef-4895-be60-b6823c11a85a'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:56:15.635919 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-530f2fc7-55ef-4895-be60-b6823c11a85a admin admin] The token cache is being invalidate because project ead596751f234d79b38d661b0d5c6343 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:56:15.636210 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-530f2fc7-55ef-4895-be60-b6823c11a85a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:56:15.636880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-530f2fc7-55ef-4895-be60-b6823c11a85a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:56:15.638606 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1300/2618] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:56:15 2022] DELETE /identity/v3/projects/ead596751f234d79b38d661b0d5c6343 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:56:19.429795 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28271e6f-9d2a-4916-adb0-bb83e79db7e7 None None] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:56:19.430654 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28271e6f-9d2a-4916-adb0-bb83e79db7e7 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:56:19.431104 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28271e6f-9d2a-4916-adb0-bb83e79db7e7 None None] PATH_INFO: `/` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:56:19.433333 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1319/2619] 10.222.0.33 () {54 vars in 921 bytes} [Mon Oct 3 12:56:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 12:56:19.444613 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c66ee-cc87-4012-b81f-20a32253360f None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:56:19.445157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c66ee-cc87-4012-b81f-20a32253360f None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:56:19.445157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c66ee-cc87-4012-b81f-20a32253360f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:56:19.483903 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-c98c66ee-cc87-4012-b81f-20a32253360f None None] MFA Rules not processed for user `eb327a8a18ca414ab8b4111a4d13aad8`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:56:19.485136 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-c98c66ee-cc87-4012-b81f-20a32253360f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:56:19.575950 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1301/2620] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Oct 3 12:56:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:56:19.591661 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28271e6f-9d2a-4916-adb0-bb83e79db7e7 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:56:19.594465 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-28271e6f-9d2a-4916-adb0-bb83e79db7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:56:19.616507 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28271e6f-9d2a-4916-adb0-bb83e79db7e7 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:56:19.629788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eafbc1-1cb0-42c8-887a-0f53feba973a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:56:19.629788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eafbc1-1cb0-42c8-887a-0f53feba973a service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:56:19.629788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eafbc1-1cb0-42c8-887a-0f53feba973a service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:56:19.629788 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eafbc1-1cb0-42c8-887a-0f53feba973a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:56:19.632275 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1cfff9-731d-4f7f-9cf0-dd6fe01ba111 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:56:19.632932 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1cfff9-731d-4f7f-9cf0-dd6fe01ba111 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:56:19.634221 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1cfff9-731d-4f7f-9cf0-dd6fe01ba111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:56:19.638681 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eafbc1-1cb0-42c8-887a-0f53feba973a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:56:19.640413 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eafbc1-1cb0-42c8-887a-0f53feba973a service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:56:19.645999 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:56:19.657434 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1320/2621] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:56:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:56:19.673400 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-1a1cfff9-731d-4f7f-9cf0-dd6fe01ba111 None None] MFA Rules not processed for user `2e6e4387a7c24bbc82c22eefe2a3d138`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:56:19.674661 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1a1cfff9-731d-4f7f-9cf0-dd6fe01ba111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:56:19.799764 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1302/2622] 10.222.0.33 () {58 vars in 967 bytes} [Mon Oct 3 12:56:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:56:19.807515 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eafbc1-1cb0-42c8-887a-0f53feba973a service neutron] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:56:19.809345 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-e7eafbc1-1cb0-42c8-887a-0f53feba973a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:56:19.831869 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422f975-4ea1-4558-945e-e983fecd5f3e None None] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:56:19.832215 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422f975-4ea1-4558-945e-e983fecd5f3e None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:56:19.832478 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422f975-4ea1-4558-945e-e983fecd5f3e None None] PATH_INFO: `/` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:56:19.834139 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1303/2623] 10.222.0.33 () {54 vars in 921 bytes} [Mon Oct 3 12:56:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 12:56:19.840538 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aeed10-0b10-4bb2-9555-102694291abf None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:56:19.840817 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aeed10-0b10-4bb2-9555-102694291abf None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:56:19.841087 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aeed10-0b10-4bb2-9555-102694291abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:56:19.877200 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-27aeed10-0b10-4bb2-9555-102694291abf None None] MFA Rules not processed for user `990f5837994f4a088c656c9b33e9a0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:56:19.879106 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-27aeed10-0b10-4bb2-9555-102694291abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:56:19.944550 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eafbc1-1cb0-42c8-887a-0f53feba973a service neutron] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:56:19.969531 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deefdc3a-ad77-4826-956a-49e7930fb0b9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b111d72a34144073917de4f566a58233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:56:19.974877 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefdc3a-ad77-4826-956a-49e7930fb0b9 service swift] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:56:19.974877 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefdc3a-ad77-4826-956a-49e7930fb0b9 service swift] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:56:19.974877 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefdc3a-ad77-4826-956a-49e7930fb0b9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:56:19.980215 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1304/2624] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Oct 3 12:56:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:56:19.982644 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deefdc3a-ad77-4826-956a-49e7930fb0b9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:56:19.986093 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aeed10-0b10-4bb2-9555-102694291abf None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:56:19.990643 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deefdc3a-ad77-4826-956a-49e7930fb0b9 service swift] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:56:19.997887 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aeed10-0b10-4bb2-9555-102694291abf None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:56:20.002281 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e34963-ddc2-404d-866c-faab3cd904d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:56:20.005819 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1321/2625] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Oct 3 12:56:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 12:56:20.006766 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e34963-ddc2-404d-866c-faab3cd904d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:56:20.006766 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e34963-ddc2-404d-866c-faab3cd904d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:56:20.006766 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e34963-ddc2-404d-866c-faab3cd904d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:56:20.021399 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e34963-ddc2-404d-866c-faab3cd904d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:56:20.024606 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e34963-ddc2-404d-866c-faab3cd904d1 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:56:20.053885 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1305/2626] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:56:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:56:20.940685 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c4705-d1d5-4d98-acfb-e359cd8fdcaf None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:56:20.941826 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c4705-d1d5-4d98-acfb-e359cd8fdcaf None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:56:20.943099 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c4705-d1d5-4d98-acfb-e359cd8fdcaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:56:21.002265 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-739c4705-d1d5-4d98-acfb-e359cd8fdcaf None None] MFA Rules not processed for user `5e29a4c0276247c286cff3eed5a1253a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:56:21.003543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-739c4705-d1d5-4d98-acfb-e359cd8fdcaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:56:21.031060 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:56:21.074571 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1322/2627] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:56:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:56:21.090527 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e34963-ddc2-404d-866c-faab3cd904d1 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:56:21.092655 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-10e34963-ddc2-404d-866c-faab3cd904d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:56:21.110157 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e34963-ddc2-404d-866c-faab3cd904d1 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:56:21.116616 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540214e8-deb3-4d66-9bbd-b97f1957323f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:56:21.117231 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540214e8-deb3-4d66-9bbd-b97f1957323f service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:56:21.123344 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540214e8-deb3-4d66-9bbd-b97f1957323f service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:56:21.123344 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540214e8-deb3-4d66-9bbd-b97f1957323f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:56:21.126359 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540214e8-deb3-4d66-9bbd-b97f1957323f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:56:21.127987 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540214e8-deb3-4d66-9bbd-b97f1957323f service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:56:21.138150 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1306/2628] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:56:21 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:56:23.959824 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7b471-2121-43af-82c0-c2e959a52ca7 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:56:23.960376 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7b471-2121-43af-82c0-c2e959a52ca7 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:56:23.960588 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7b471-2121-43af-82c0-c2e959a52ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:56:23.981862 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-70d7b471-2121-43af-82c0-c2e959a52ca7 None None] MFA Rules not processed for user `5e29a4c0276247c286cff3eed5a1253a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:56:23.983073 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-70d7b471-2121-43af-82c0-c2e959a52ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:56:23.991007 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1323/2629] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:56:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:56:31.133885 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=67 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:56:33.976629 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=19 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:57:04.625933 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540214e8-deb3-4d66-9bbd-b97f1957323f service nova] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:57:04.627342 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-540214e8-deb3-4d66-9bbd-b97f1957323f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:57:04.639664 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540214e8-deb3-4d66-9bbd-b97f1957323f service nova] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:57:04.643667 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dce66bd-6ee9-4a82-ae17-fec56f9be7c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:57:04.644304 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dce66bd-6ee9-4a82-ae17-fec56f9be7c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:57:04.644608 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dce66bd-6ee9-4a82-ae17-fec56f9be7c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:57:04.644877 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dce66bd-6ee9-4a82-ae17-fec56f9be7c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:57:04.646530 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-2dce66bd-6ee9-4a82-ae17-fec56f9be7c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:57:04.708712 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dce66bd-6ee9-4a82-ae17-fec56f9be7c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:57:04.710148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dce66bd-6ee9-4a82-ae17-fec56f9be7c6 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:57:04.720161 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1307/2630] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:57:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:57:08.622693 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d7b471-2121-43af-82c0-c2e959a52ca7 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:57:08.624150 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-70d7b471-2121-43af-82c0-c2e959a52ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:57:08.638338 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d7b471-2121-43af-82c0-c2e959a52ca7 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:57:08.643431 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2976a7-7d40-4fab-a952-8b21a694afb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:57:08.644073 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2976a7-7d40-4fab-a952-8b21a694afb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:57:08.644350 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2976a7-7d40-4fab-a952-8b21a694afb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:57:08.644621 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2976a7-7d40-4fab-a952-8b21a694afb1 admin admin] PATH_INFO: `/v3/users/676b2a6bbf6e4df1b47dba2a9b8e8862` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:57:08.645785 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2976a7-7d40-4fab-a952-8b21a694afb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=676b2a6bbf6e4df1b47dba2a9b8e8862)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:57:08.647744 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2976a7-7d40-4fab-a952-8b21a694afb1 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:57:08.834510 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-ab2976a7-7d40-4fab-a952-8b21a694afb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '676b2a6bbf6e4df1b47dba2a9b8e8862'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:57:08.843182 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-ab2976a7-7d40-4fab-a952-8b21a694afb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '676b2a6bbf6e4df1b47dba2a9b8e8862'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:57:08.856767 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-ab2976a7-7d40-4fab-a952-8b21a694afb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '676b2a6bbf6e4df1b47dba2a9b8e8862'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:57:08.857562 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-ab2976a7-7d40-4fab-a952-8b21a694afb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '676b2a6bbf6e4df1b47dba2a9b8e8862'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:57:08.940667 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-ab2976a7-7d40-4fab-a952-8b21a694afb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '676b2a6bbf6e4df1b47dba2a9b8e8862'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:57:08.947576 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1324/2631] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:57:08 2022] DELETE /identity/v3/users/676b2a6bbf6e4df1b47dba2a9b8e8862 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:57:09.222765 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dce66bd-6ee9-4a82-ae17-fec56f9be7c6 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:57:09.234952 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dce66bd-6ee9-4a82-ae17-fec56f9be7c6 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:57:09.299519 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f9960-cce5-4df6-bc09-660d4db7d4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:57:09.300160 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f9960-cce5-4df6-bc09-660d4db7d4a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:57:09.305102 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f9960-cce5-4df6-bc09-660d4db7d4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:57:09.305102 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f9960-cce5-4df6-bc09-660d4db7d4a5 admin admin] PATH_INFO: `/v3/projects/0e20f793bd0b46a59df730183818d9f8` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:57:09.305102 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f9960-cce5-4df6-bc09-660d4db7d4a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e20f793bd0b46a59df730183818d9f8)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:57:09.305102 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f9960-cce5-4df6-bc09-660d4db7d4a5 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:57:09.446395 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a10f9960-cce5-4df6-bc09-660d4db7d4a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e20f793bd0b46a59df730183818d9f8', 'request_id': 'req-a10f9960-cce5-4df6-bc09-660d4db7d4a5'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:57:09.451866 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a10f9960-cce5-4df6-bc09-660d4db7d4a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e20f793bd0b46a59df730183818d9f8', 'request_id': 'req-a10f9960-cce5-4df6-bc09-660d4db7d4a5'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:57:09.500740 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a10f9960-cce5-4df6-bc09-660d4db7d4a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e20f793bd0b46a59df730183818d9f8', 'request_id': 'req-a10f9960-cce5-4df6-bc09-660d4db7d4a5'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:57:09.513233 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a10f9960-cce5-4df6-bc09-660d4db7d4a5 admin admin] The token cache is being invalidate because project 0e20f793bd0b46a59df730183818d9f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:57:09.513534 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a10f9960-cce5-4df6-bc09-660d4db7d4a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:57:09.514231 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a10f9960-cce5-4df6-bc09-660d4db7d4a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:57:09.515840 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1308/2632] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:57:09 2022] DELETE /identity/v3/projects/0e20f793bd0b46a59df730183818d9f8 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:57:09.520752 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e86947-cd10-4fe9-b3c1-2328fe89d531 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:57:09.521042 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e86947-cd10-4fe9-b3c1-2328fe89d531 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:57:09.521322 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e86947-cd10-4fe9-b3c1-2328fe89d531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:57:09.541816 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-00e86947-cd10-4fe9-b3c1-2328fe89d531 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:57:09.543016 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-00e86947-cd10-4fe9-b3c1-2328fe89d531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:57:09.606603 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1325/2633] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:57:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:57:09.614683 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa3116-9433-4dec-abf9-5a65c95b195b None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:57:09.615061 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa3116-9433-4dec-abf9-5a65c95b195b None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:57:09.615370 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa3116-9433-4dec-abf9-5a65c95b195b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:57:09.636650 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-d3aa3116-9433-4dec-abf9-5a65c95b195b None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:57:09.637850 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d3aa3116-9433-4dec-abf9-5a65c95b195b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:57:09.645755 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1309/2634] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:57:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:57:09.650412 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e86947-cd10-4fe9-b3c1-2328fe89d531 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:57:09.659987 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e86947-cd10-4fe9-b3c1-2328fe89d531 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:57:09.665167 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ec10b-8fd9-4b51-9dde-b93d091437c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:57:09.665766 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ec10b-8fd9-4b51-9dde-b93d091437c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:57:09.666048 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ec10b-8fd9-4b51-9dde-b93d091437c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:57:09.666323 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ec10b-8fd9-4b51-9dde-b93d091437c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:57:09.666964 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ec10b-8fd9-4b51-9dde-b93d091437c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:57:09.667269 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ec10b-8fd9-4b51-9dde-b93d091437c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:57:09.668876 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ec10b-8fd9-4b51-9dde-b93d091437c8 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:57:09.680264 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1326/2635] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:57:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:57:09.684595 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aa3116-9433-4dec-abf9-5a65c95b195b None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:57:09.697132 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aa3116-9433-4dec-abf9-5a65c95b195b None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:57:09.701776 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b347347-7e8f-4978-abf3-e1d0312d86f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:57:09.702437 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b347347-7e8f-4978-abf3-e1d0312d86f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:57:09.702724 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b347347-7e8f-4978-abf3-e1d0312d86f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:57:09.703000 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b347347-7e8f-4978-abf3-e1d0312d86f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:57:09.703691 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b347347-7e8f-4978-abf3-e1d0312d86f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:57:09.705595 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b347347-7e8f-4978-abf3-e1d0312d86f9 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:57:09.721720 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:57:09.826855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1310/2636] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:57:09 2022] POST /identity/v3/projects => generated 373 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:57:09.831081 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ec10b-8fd9-4b51-9dde-b93d091437c8 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:57:09.841747 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ec10b-8fd9-4b51-9dde-b93d091437c8 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:57:09.898400 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423bb0d1-8dba-4f89-aff7-7244335ec758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:57:09.899603 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423bb0d1-8dba-4f89-aff7-7244335ec758 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:57:09.899938 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423bb0d1-8dba-4f89-aff7-7244335ec758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:57:09.900158 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423bb0d1-8dba-4f89-aff7-7244335ec758 admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:57:09.900798 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423bb0d1-8dba-4f89-aff7-7244335ec758 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:57:09.902725 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423bb0d1-8dba-4f89-aff7-7244335ec758 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:57:09.910892 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:57:09.974173 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1327/2637] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:57:09 2022] POST /identity/v3/users => generated 287 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:57:09.978621 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b347347-7e8f-4978-abf3-e1d0312d86f9 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:57:09.989086 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b347347-7e8f-4978-abf3-e1d0312d86f9 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:57:09.994412 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f2691-5380-4277-84b8-4dcbb6d2afbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:57:09.995012 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f2691-5380-4277-84b8-4dcbb6d2afbf admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:57:09.995291 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f2691-5380-4277-84b8-4dcbb6d2afbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:57:09.995592 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f2691-5380-4277-84b8-4dcbb6d2afbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:57:09.996230 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203f2691-5380-4277-84b8-4dcbb6d2afbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:57:09.997855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203f2691-5380-4277-84b8-4dcbb6d2afbf admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:57:10.007791 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1311/2638] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:57:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:57:10.013298 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423bb0d1-8dba-4f89-aff7-7244335ec758 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:57:10.026924 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423bb0d1-8dba-4f89-aff7-7244335ec758 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:57:10.032895 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073652aa-79e9-49b6-83f9-53593e5f31dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:57:10.033582 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073652aa-79e9-49b6-83f9-53593e5f31dd admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:57:10.033900 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073652aa-79e9-49b6-83f9-53593e5f31dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:57:10.034186 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073652aa-79e9-49b6-83f9-53593e5f31dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:57:10.034905 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073652aa-79e9-49b6-83f9-53593e5f31dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:57:10.036801 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073652aa-79e9-49b6-83f9-53593e5f31dd admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:57:10.049172 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1328/2639] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:57:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:57:10.052945 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f2691-5380-4277-84b8-4dcbb6d2afbf admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:57:10.063085 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f2691-5380-4277-84b8-4dcbb6d2afbf admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:57:10.067695 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae3403-cbb7-4058-ab5b-9fba0721c359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:57:10.068332 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae3403-cbb7-4058-ab5b-9fba0721c359 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:57:10.068610 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae3403-cbb7-4058-ab5b-9fba0721c359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:57:10.068886 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae3403-cbb7-4058-ab5b-9fba0721c359 admin admin] PATH_INFO: `/v3/projects/e2082fde4f954d60b1d3c64ad855ef91/users/6ddb6858a45847f994d3be5df4ce644d/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:57:10.084406 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ae3403-cbb7-4058-ab5b-9fba0721c359 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2082fde4f954d60b1d3c64ad855ef91, user_id=6ddb6858a45847f994d3be5df4ce644d, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:57:10.087825 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ae3403-cbb7-4058-ab5b-9fba0721c359 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:57:10.157984 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1312/2640] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:57:10 2022] PUT /identity/v3/projects/e2082fde4f954d60b1d3c64ad855ef91/users/6ddb6858a45847f994d3be5df4ce644d/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:57:10.168974 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e51a1-3be4-4b12-9b93-3ebdbde3579d None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:57:10.169265 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e51a1-3be4-4b12-9b93-3ebdbde3579d None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:57:10.169540 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e51a1-3be4-4b12-9b93-3ebdbde3579d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:57:10.197867 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-356e51a1-3be4-4b12-9b93-3ebdbde3579d None None] MFA Rules not processed for user `6ddb6858a45847f994d3be5df4ce644d`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:57:10.199097 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-356e51a1-3be4-4b12-9b93-3ebdbde3579d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:57:10.266887 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1329/2641] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:57:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:57:10.271406 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae3403-cbb7-4058-ab5b-9fba0721c359 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:57:10.334459 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae3403-cbb7-4058-ab5b-9fba0721c359 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:57:10.339451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47abc8d9-6fd5-4b20-b38a-5f35c3f59bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:57:10.340108 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47abc8d9-6fd5-4b20-b38a-5f35c3f59bc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:57:10.340350 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47abc8d9-6fd5-4b20-b38a-5f35c3f59bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:57:10.340656 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47abc8d9-6fd5-4b20-b38a-5f35c3f59bc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:57:10.341293 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47abc8d9-6fd5-4b20-b38a-5f35c3f59bc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:57:10.343240 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47abc8d9-6fd5-4b20-b38a-5f35c3f59bc9 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:57:10.450723 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1313/2642] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:57:10 2022] POST /identity/v3/projects => generated 373 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:57:10.454926 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e51a1-3be4-4b12-9b93-3ebdbde3579d None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:57:10.467723 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e51a1-3be4-4b12-9b93-3ebdbde3579d None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:57:10.526379 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c629132-94e6-4588-8561-3f9d8eb73a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:57:10.526908 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c629132-94e6-4588-8561-3f9d8eb73a22 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:57:10.527230 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c629132-94e6-4588-8561-3f9d8eb73a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:57:10.527529 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c629132-94e6-4588-8561-3f9d8eb73a22 admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:57:10.528125 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c629132-94e6-4588-8561-3f9d8eb73a22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:57:10.530023 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c629132-94e6-4588-8561-3f9d8eb73a22 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:57:10.667245 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1330/2643] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:57:10 2022] POST /identity/v3/users => generated 293 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:57:10.671215 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47abc8d9-6fd5-4b20-b38a-5f35c3f59bc9 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:57:10.682441 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47abc8d9-6fd5-4b20-b38a-5f35c3f59bc9 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:57:10.689140 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71703876-80d0-4e20-8ca7-c9301c7fc859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:57:10.689140 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71703876-80d0-4e20-8ca7-c9301c7fc859 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:57:10.689140 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71703876-80d0-4e20-8ca7-c9301c7fc859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:57:10.689140 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71703876-80d0-4e20-8ca7-c9301c7fc859 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:57:10.689551 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71703876-80d0-4e20-8ca7-c9301c7fc859 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:57:10.690430 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71703876-80d0-4e20-8ca7-c9301c7fc859 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:57:10.702077 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1314/2644] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:57:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:57:10.706243 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c629132-94e6-4588-8561-3f9d8eb73a22 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:57:10.716600 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c629132-94e6-4588-8561-3f9d8eb73a22 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:57:10.721540 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab5502-0c21-4475-a95b-1aabe18c77e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:57:10.722208 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab5502-0c21-4475-a95b-1aabe18c77e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:57:10.722667 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab5502-0c21-4475-a95b-1aabe18c77e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:57:10.723132 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab5502-0c21-4475-a95b-1aabe18c77e3 admin admin] PATH_INFO: `/v3/projects/a87c76bbc1b34c6391f14434c63eac4e/users/3e7b7fda891148029eb562e328f8ac40/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:57:10.746932 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab5502-0c21-4475-a95b-1aabe18c77e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a87c76bbc1b34c6391f14434c63eac4e, user_id=3e7b7fda891148029eb562e328f8ac40, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:57:10.749650 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab5502-0c21-4475-a95b-1aabe18c77e3 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:57:10.810381 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1331/2645] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:57:10 2022] PUT /identity/v3/projects/a87c76bbc1b34c6391f14434c63eac4e/users/3e7b7fda891148029eb562e328f8ac40/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:57:10.814310 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71703876-80d0-4e20-8ca7-c9301c7fc859 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:57:10.825347 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71703876-80d0-4e20-8ca7-c9301c7fc859 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:57:10.883626 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c191dd-e7df-4452-9df1-4cc72f7602bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:57:10.884238 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c191dd-e7df-4452-9df1-4cc72f7602bf admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:57:10.884517 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c191dd-e7df-4452-9df1-4cc72f7602bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:57:10.884789 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c191dd-e7df-4452-9df1-4cc72f7602bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:57:10.885422 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c191dd-e7df-4452-9df1-4cc72f7602bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:57:10.887036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c191dd-e7df-4452-9df1-4cc72f7602bf admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:57:10.896631 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1315/2646] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:57:10 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 12:57:10.900956 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab5502-0c21-4475-a95b-1aabe18c77e3 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:57:10.912128 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab5502-0c21-4475-a95b-1aabe18c77e3 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:57:10.919608 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429c676-3b0f-4249-8318-c2082bab44af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:57:10.920483 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429c676-3b0f-4249-8318-c2082bab44af admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:57:10.920898 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429c676-3b0f-4249-8318-c2082bab44af admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:57:10.921380 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429c676-3b0f-4249-8318-c2082bab44af admin admin] PATH_INFO: `/v3/projects/a87c76bbc1b34c6391f14434c63eac4e/users/3e7b7fda891148029eb562e328f8ac40/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:57:10.924305 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6429c676-3b0f-4249-8318-c2082bab44af admin admin] RBAC: Authorizing `identity:create_grant(project_id=a87c76bbc1b34c6391f14434c63eac4e, user_id=3e7b7fda891148029eb562e328f8ac40, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:57:10.928494 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6429c676-3b0f-4249-8318-c2082bab44af admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:57:10.939555 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1332/2647] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:57:10 2022] PUT /identity/v3/projects/a87c76bbc1b34c6391f14434c63eac4e/users/3e7b7fda891148029eb562e328f8ac40/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:57:10.951850 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c84159-f86f-44ff-acb8-3535bb765ab1 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:57:10.952135 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c84159-f86f-44ff-acb8-3535bb765ab1 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:57:10.952406 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c84159-f86f-44ff-acb8-3535bb765ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:57:10.981138 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-b1c84159-f86f-44ff-acb8-3535bb765ab1 None None] MFA Rules not processed for user `3e7b7fda891148029eb562e328f8ac40`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:57:10.982328 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b1c84159-f86f-44ff-acb8-3535bb765ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:57:11.001053 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:57:11.052330 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1316/2648] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:57:10 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:57:11.064832 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429c676-3b0f-4249-8318-c2082bab44af admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:57:11.066346 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-6429c676-3b0f-4249-8318-c2082bab44af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:57:11.091555 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:57:11.128708 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429c676-3b0f-4249-8318-c2082bab44af admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:57:11.132679 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f370c-8d71-4710-994c-039e39a28fe3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:57:11.133441 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f370c-8d71-4710-994c-039e39a28fe3 service cinder] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:57:11.133635 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f370c-8d71-4710-994c-039e39a28fe3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:57:11.133912 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f370c-8d71-4710-994c-039e39a28fe3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:57:11.143444 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f370c-8d71-4710-994c-039e39a28fe3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:57:11.144864 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f370c-8d71-4710-994c-039e39a28fe3 service cinder] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:57:11.154714 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1333/2649] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:57:11 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:57:11.344814 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c84159-f86f-44ff-acb8-3535bb765ab1 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:57:11.355586 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c84159-f86f-44ff-acb8-3535bb765ab1 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:57:11.359199 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23558ee5-50a8-4691-85ce-dd545beb1f1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:57:11.359866 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23558ee5-50a8-4691-85ce-dd545beb1f1c service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:57:11.360146 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23558ee5-50a8-4691-85ce-dd545beb1f1c service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:57:11.360420 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23558ee5-50a8-4691-85ce-dd545beb1f1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:57:11.419896 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23558ee5-50a8-4691-85ce-dd545beb1f1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:57:11.421340 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23558ee5-50a8-4691-85ce-dd545beb1f1c service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:57:11.430619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1317/2650] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:57:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:57:21.150624 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=17 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:57:21.426301 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=48 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:57:28.484587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25e752-5725-41ad-afe8-b29c688dbb76 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:57:28.485114 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25e752-5725-41ad-afe8-b29c688dbb76 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:57:28.485114 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25e752-5725-41ad-afe8-b29c688dbb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:57:28.516379 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-0d25e752-5725-41ad-afe8-b29c688dbb76 None None] MFA Rules not processed for user `5e29a4c0276247c286cff3eed5a1253a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:57:28.517800 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-0d25e752-5725-41ad-afe8-b29c688dbb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:57:28.612800 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1334/2651] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:57:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:57:35.063425 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23558ee5-50a8-4691-85ce-dd545beb1f1c service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:57:35.064831 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-23558ee5-50a8-4691-85ce-dd545beb1f1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:57:35.103428 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23558ee5-50a8-4691-85ce-dd545beb1f1c service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:57:35.109169 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac1bd7a-ad09-4d51-9bc8-471cc83ba108 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:57:35.109685 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1bd7a-ad09-4d51-9bc8-471cc83ba108 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:57:35.109963 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1bd7a-ad09-4d51-9bc8-471cc83ba108 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:57:35.110382 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1bd7a-ad09-4d51-9bc8-471cc83ba108 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:57:35.112168 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-8ac1bd7a-ad09-4d51-9bc8-471cc83ba108 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:57:35.177552 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac1bd7a-ad09-4d51-9bc8-471cc83ba108 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:57:35.178992 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac1bd7a-ad09-4d51-9bc8-471cc83ba108 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:57:35.188714 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1318/2652] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:57:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:57:38.610483 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=37 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:57:41.836844 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d25e752-5725-41ad-afe8-b29c688dbb76 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:57:41.838243 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-0d25e752-5725-41ad-afe8-b29c688dbb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:57:41.934126 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d25e752-5725-41ad-afe8-b29c688dbb76 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:57:41.939872 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4752970-3fab-494d-a36a-06639243bd80 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25eef64e92e9422ea62508c3eb0a197e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:57:41.940476 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4752970-3fab-494d-a36a-06639243bd80 service placement] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:57:41.940758 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4752970-3fab-494d-a36a-06639243bd80 service placement] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:57:41.941029 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4752970-3fab-494d-a36a-06639243bd80 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:57:41.942741 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-b4752970-3fab-494d-a36a-06639243bd80 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:57:41.955081 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4752970-3fab-494d-a36a-06639243bd80 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:57:41.956685 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4752970-3fab-494d-a36a-06639243bd80 service placement] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:57:41.966368 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1335/2653] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:57:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:57:45.184423 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=37 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:57:47.885207 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb2baa-6a5e-46f8-9086-7fe4112a88e5 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:57:47.885767 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb2baa-6a5e-46f8-9086-7fe4112a88e5 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:57:47.885767 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb2baa-6a5e-46f8-9086-7fe4112a88e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:57:47.907394 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-65fb2baa-6a5e-46f8-9086-7fe4112a88e5 None None] MFA Rules not processed for user `7ffffa27986f461ebc10c23c3c9817f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:57:47.908619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-65fb2baa-6a5e-46f8-9086-7fe4112a88e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:57:47.916154 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1319/2654] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:57:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:57:48.346374 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4752970-3fab-494d-a36a-06639243bd80 service placement] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:57:48.347882 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-b4752970-3fab-494d-a36a-06639243bd80 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:57:48.429048 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4752970-3fab-494d-a36a-06639243bd80 service placement] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:57:48.433051 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65af625-5c0a-4591-b7d8-bf6f64dffff3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:57:48.433659 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65af625-5c0a-4591-b7d8-bf6f64dffff3 service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:57:48.438672 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65af625-5c0a-4591-b7d8-bf6f64dffff3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:57:48.438672 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65af625-5c0a-4591-b7d8-bf6f64dffff3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:57:48.438672 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-a65af625-5c0a-4591-b7d8-bf6f64dffff3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:57:48.448251 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65af625-5c0a-4591-b7d8-bf6f64dffff3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:57:48.449831 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65af625-5c0a-4591-b7d8-bf6f64dffff3 service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:57:48.458902 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1336/2655] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:57:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:57:57.900600 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:57:58.455084 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=79 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:03.688751 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fb2baa-6a5e-46f8-9086-7fe4112a88e5 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:03.785418 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fb2baa-6a5e-46f8-9086-7fe4112a88e5 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:03.790294 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c995a6-d27b-41b9-bfbc-d285a1fd0343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:03.790946 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c995a6-d27b-41b9-bfbc-d285a1fd0343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:03.791221 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c995a6-d27b-41b9-bfbc-d285a1fd0343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:03.791527 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c995a6-d27b-41b9-bfbc-d285a1fd0343 admin admin] PATH_INFO: `/v3/users/6ddb6858a45847f994d3be5df4ce644d` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:03.792668 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c995a6-d27b-41b9-bfbc-d285a1fd0343 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ddb6858a45847f994d3be5df4ce644d)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:03.794842 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c995a6-d27b-41b9-bfbc-d285a1fd0343 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:03.978191 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c5c995a6-d27b-41b9-bfbc-d285a1fd0343 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ddb6858a45847f994d3be5df4ce644d'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:03.991010 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c5c995a6-d27b-41b9-bfbc-d285a1fd0343 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ddb6858a45847f994d3be5df4ce644d'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:03.992020 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c5c995a6-d27b-41b9-bfbc-d285a1fd0343 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ddb6858a45847f994d3be5df4ce644d'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:04.001159 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c5c995a6-d27b-41b9-bfbc-d285a1fd0343 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ddb6858a45847f994d3be5df4ce644d'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:04.054193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c5c995a6-d27b-41b9-bfbc-d285a1fd0343 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ddb6858a45847f994d3be5df4ce644d'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:04.061838 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1320/2656] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:58:03 2022] DELETE /identity/v3/users/6ddb6858a45847f994d3be5df4ce644d => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:04.066547 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1be3cd-21fb-4755-92fd-334575c413b6 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:04.067577 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1be3cd-21fb-4755-92fd-334575c413b6 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:04.067888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1be3cd-21fb-4755-92fd-334575c413b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:04.109781 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-ee1be3cd-21fb-4755-92fd-334575c413b6 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:58:04.110992 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-ee1be3cd-21fb-4755-92fd-334575c413b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:04.175560 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1337/2657] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:58:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:58:04.185264 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c995a6-d27b-41b9-bfbc-d285a1fd0343 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:04.186798 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-c5c995a6-d27b-41b9-bfbc-d285a1fd0343 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:04.245794 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c995a6-d27b-41b9-bfbc-d285a1fd0343 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:04.249801 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c999932a-86d8-4a17-8694-08e11568c737 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:04.250437 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c999932a-86d8-4a17-8694-08e11568c737 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:04.250735 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c999932a-86d8-4a17-8694-08e11568c737 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:04.250991 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c999932a-86d8-4a17-8694-08e11568c737 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:04.260947 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c999932a-86d8-4a17-8694-08e11568c737 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:04.264197 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c999932a-86d8-4a17-8694-08e11568c737 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:04.272492 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1321/2658] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:58:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:58:04.751908 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1be3cd-21fb-4755-92fd-334575c413b6 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:04.753764 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-ee1be3cd-21fb-4755-92fd-334575c413b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:04.768121 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1be3cd-21fb-4755-92fd-334575c413b6 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:04.785741 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640727d6-d54c-44c6-bd14-960149ee5225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:04.789254 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640727d6-d54c-44c6-bd14-960149ee5225 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:04.789254 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640727d6-d54c-44c6-bd14-960149ee5225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:04.789254 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640727d6-d54c-44c6-bd14-960149ee5225 admin admin] PATH_INFO: `/v3/projects/e2082fde4f954d60b1d3c64ad855ef91` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:04.789254 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640727d6-d54c-44c6-bd14-960149ee5225 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2082fde4f954d60b1d3c64ad855ef91)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:04.790083 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640727d6-d54c-44c6-bd14-960149ee5225 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:04.905947 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-640727d6-d54c-44c6-bd14-960149ee5225 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2082fde4f954d60b1d3c64ad855ef91', 'request_id': 'req-640727d6-d54c-44c6-bd14-960149ee5225'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:04.912384 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-640727d6-d54c-44c6-bd14-960149ee5225 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2082fde4f954d60b1d3c64ad855ef91', 'request_id': 'req-640727d6-d54c-44c6-bd14-960149ee5225'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:04.973347 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-640727d6-d54c-44c6-bd14-960149ee5225 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2082fde4f954d60b1d3c64ad855ef91', 'request_id': 'req-640727d6-d54c-44c6-bd14-960149ee5225'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:04.985094 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-640727d6-d54c-44c6-bd14-960149ee5225 admin admin] The token cache is being invalidate because project e2082fde4f954d60b1d3c64ad855ef91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:58:04.985386 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-640727d6-d54c-44c6-bd14-960149ee5225 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:04.986024 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-640727d6-d54c-44c6-bd14-960149ee5225 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:04.987569 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1338/2659] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:58:04 2022] DELETE /identity/v3/projects/e2082fde4f954d60b1d3c64ad855ef91 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:58:04.991924 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c999932a-86d8-4a17-8694-08e11568c737 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:04.998465 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:05.002290 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c999932a-86d8-4a17-8694-08e11568c737 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:05.060878 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c81fef-09a1-46bf-a0df-f437553d8a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:05.061572 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c81fef-09a1-46bf-a0df-f437553d8a82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:05.061855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c81fef-09a1-46bf-a0df-f437553d8a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:05.062130 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c81fef-09a1-46bf-a0df-f437553d8a82 admin admin] PATH_INFO: `/v3/users/3e7b7fda891148029eb562e328f8ac40` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:05.063249 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c81fef-09a1-46bf-a0df-f437553d8a82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e7b7fda891148029eb562e328f8ac40)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:05.065289 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c81fef-09a1-46bf-a0df-f437553d8a82 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:05.261884 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b6c81fef-09a1-46bf-a0df-f437553d8a82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e7b7fda891148029eb562e328f8ac40'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:05.274605 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b6c81fef-09a1-46bf-a0df-f437553d8a82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e7b7fda891148029eb562e328f8ac40'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:05.275302 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b6c81fef-09a1-46bf-a0df-f437553d8a82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e7b7fda891148029eb562e328f8ac40'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:05.282731 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b6c81fef-09a1-46bf-a0df-f437553d8a82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e7b7fda891148029eb562e328f8ac40'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:05.356955 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b6c81fef-09a1-46bf-a0df-f437553d8a82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e7b7fda891148029eb562e328f8ac40'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:05.363649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1322/2660] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:58:04 2022] DELETE /identity/v3/users/3e7b7fda891148029eb562e328f8ac40 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:05.921120 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640727d6-d54c-44c6-bd14-960149ee5225 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:05.922919 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-640727d6-d54c-44c6-bd14-960149ee5225 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:05.936332 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640727d6-d54c-44c6-bd14-960149ee5225 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:06.011455 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43de1f5-6e07-4087-af38-e630929fe177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:06.012098 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43de1f5-6e07-4087-af38-e630929fe177 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:06.012380 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43de1f5-6e07-4087-af38-e630929fe177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:06.012785 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43de1f5-6e07-4087-af38-e630929fe177 admin admin] PATH_INFO: `/v3/projects/a87c76bbc1b34c6391f14434c63eac4e` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:06.013973 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43de1f5-6e07-4087-af38-e630929fe177 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a87c76bbc1b34c6391f14434c63eac4e)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:06.015976 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43de1f5-6e07-4087-af38-e630929fe177 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:06.024294 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:06.157588 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-f43de1f5-6e07-4087-af38-e630929fe177 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a87c76bbc1b34c6391f14434c63eac4e', 'request_id': 'req-f43de1f5-6e07-4087-af38-e630929fe177'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:06.163802 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-f43de1f5-6e07-4087-af38-e630929fe177 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a87c76bbc1b34c6391f14434c63eac4e', 'request_id': 'req-f43de1f5-6e07-4087-af38-e630929fe177'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:06.250515 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-f43de1f5-6e07-4087-af38-e630929fe177 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a87c76bbc1b34c6391f14434c63eac4e', 'request_id': 'req-f43de1f5-6e07-4087-af38-e630929fe177'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:06.260571 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c81fef-09a1-46bf-a0df-f437553d8a82 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:06.261854 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-f43de1f5-6e07-4087-af38-e630929fe177 admin admin] The token cache is being invalidate because project a87c76bbc1b34c6391f14434c63eac4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:58:06.261982 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b6c81fef-09a1-46bf-a0df-f437553d8a82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:06.262150 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-f43de1f5-6e07-4087-af38-e630929fe177 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:06.262929 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-f43de1f5-6e07-4087-af38-e630929fe177 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:06.264727 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1339/2661] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:58:05 2022] DELETE /identity/v3/projects/a87c76bbc1b34c6391f14434c63eac4e => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:06.269449 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d7b2b-c3f7-498e-b8ed-a86f8ea5a992 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:06.269758 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d7b2b-c3f7-498e-b8ed-a86f8ea5a992 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:06.270035 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d7b2b-c3f7-498e-b8ed-a86f8ea5a992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:06.320428 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-9b7d7b2b-c3f7-498e-b8ed-a86f8ea5a992 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:58:06.322110 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-9b7d7b2b-c3f7-498e-b8ed-a86f8ea5a992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:06.322756 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c81fef-09a1-46bf-a0df-f437553d8a82 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:06.363385 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:06.375832 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec70bfe-3e35-464a-9c8d-30cf6fdbbcfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:06.376534 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec70bfe-3e35-464a-9c8d-30cf6fdbbcfb service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:06.376822 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec70bfe-3e35-464a-9c8d-30cf6fdbbcfb service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:06.377101 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec70bfe-3e35-464a-9c8d-30cf6fdbbcfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:06.379031 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-0ec70bfe-3e35-464a-9c8d-30cf6fdbbcfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:06.384397 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1340/2662] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:58:06.392313 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec70bfe-3e35-464a-9c8d-30cf6fdbbcfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:06.393677 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42391703-ec40-43ff-8863-a6bd315b0ad0 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:06.393815 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec70bfe-3e35-464a-9c8d-30cf6fdbbcfb service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:06.393965 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42391703-ec40-43ff-8863-a6bd315b0ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:06.394245 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42391703-ec40-43ff-8863-a6bd315b0ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:06.404607 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1323/2663] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:58:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:58:06.414992 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-42391703-ec40-43ff-8863-a6bd315b0ad0 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:58:06.416232 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-42391703-ec40-43ff-8863-a6bd315b0ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:06.425452 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1341/2664] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:58:06.432645 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec70bfe-3e35-464a-9c8d-30cf6fdbbcfb service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:06.442394 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec70bfe-3e35-464a-9c8d-30cf6fdbbcfb service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:06.447189 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99117b73-951a-4037-aac6-0ffe6c83a13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:06.448197 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99117b73-951a-4037-aac6-0ffe6c83a13c admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:06.448774 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99117b73-951a-4037-aac6-0ffe6c83a13c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:06.449040 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99117b73-951a-4037-aac6-0ffe6c83a13c admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:06.449679 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99117b73-951a-4037-aac6-0ffe6c83a13c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:58:06.449978 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99117b73-951a-4037-aac6-0ffe6c83a13c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:06.451830 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99117b73-951a-4037-aac6-0ffe6c83a13c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:06.463337 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1324/2665] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:58:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:58:06.468313 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42391703-ec40-43ff-8863-a6bd315b0ad0 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:06.477710 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42391703-ec40-43ff-8863-a6bd315b0ad0 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:06.483143 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824b97a0-c1bd-49b3-ac7f-b380b11e80cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:06.483781 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b97a0-c1bd-49b3-ac7f-b380b11e80cf admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:06.484065 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b97a0-c1bd-49b3-ac7f-b380b11e80cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:06.484338 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b97a0-c1bd-49b3-ac7f-b380b11e80cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:06.484975 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824b97a0-c1bd-49b3-ac7f-b380b11e80cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:06.486906 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824b97a0-c1bd-49b3-ac7f-b380b11e80cf admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:06.617127 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1342/2666] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:58:06 2022] POST /identity/v3/projects => generated 371 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:58:06.621384 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99117b73-951a-4037-aac6-0ffe6c83a13c admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:06.631374 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99117b73-951a-4037-aac6-0ffe6c83a13c admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:06.690426 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b4578-ebfe-4fcf-b818-6dee09c1e18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:06.691044 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b4578-ebfe-4fcf-b818-6dee09c1e18a admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:06.691347 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b4578-ebfe-4fcf-b818-6dee09c1e18a admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:06.691615 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b4578-ebfe-4fcf-b818-6dee09c1e18a admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:06.692241 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b4578-ebfe-4fcf-b818-6dee09c1e18a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:06.694188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b4578-ebfe-4fcf-b818-6dee09c1e18a admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:06.791208 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1325/2667] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:58:06 2022] POST /identity/v3/users => generated 286 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:58:06.796500 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824b97a0-c1bd-49b3-ac7f-b380b11e80cf admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:06.816613 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824b97a0-c1bd-49b3-ac7f-b380b11e80cf admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:06.822373 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64865423-5544-4075-86f8-9744c9cfdf81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:06.822999 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64865423-5544-4075-86f8-9744c9cfdf81 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:06.823279 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64865423-5544-4075-86f8-9744c9cfdf81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:06.823592 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64865423-5544-4075-86f8-9744c9cfdf81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:06.824234 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64865423-5544-4075-86f8-9744c9cfdf81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:06.826931 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64865423-5544-4075-86f8-9744c9cfdf81 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:06.839958 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1343/2668] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:58:06.842984 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b4578-ebfe-4fcf-b818-6dee09c1e18a admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:06.853295 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b4578-ebfe-4fcf-b818-6dee09c1e18a admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:06.859085 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0383e38c-45b4-4eaa-b5a4-928145b3f696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:06.859733 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383e38c-45b4-4eaa-b5a4-928145b3f696 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:06.860173 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383e38c-45b4-4eaa-b5a4-928145b3f696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:06.860456 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383e38c-45b4-4eaa-b5a4-928145b3f696 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:06.861201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0383e38c-45b4-4eaa-b5a4-928145b3f696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:06.862794 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0383e38c-45b4-4eaa-b5a4-928145b3f696 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:06.880647 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1326/2669] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:58:06.881794 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64865423-5544-4075-86f8-9744c9cfdf81 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:06.896073 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64865423-5544-4075-86f8-9744c9cfdf81 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:06.901267 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d5f39d-ee6a-4054-833b-d7c16c4d3ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:06.902283 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d5f39d-ee6a-4054-833b-d7c16c4d3ee5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:06.902708 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d5f39d-ee6a-4054-833b-d7c16c4d3ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:06.902990 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d5f39d-ee6a-4054-833b-d7c16c4d3ee5 admin admin] PATH_INFO: `/v3/projects/be44ac233d084dbf87abe5e0f15bb020/users/c506636f5f984fb68c11c2cc77bc0aa4/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:06.918573 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d5f39d-ee6a-4054-833b-d7c16c4d3ee5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be44ac233d084dbf87abe5e0f15bb020, user_id=c506636f5f984fb68c11c2cc77bc0aa4, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:06.921791 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d5f39d-ee6a-4054-833b-d7c16c4d3ee5 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:06.974208 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1344/2670] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:58:06 2022] PUT /identity/v3/projects/be44ac233d084dbf87abe5e0f15bb020/users/c506636f5f984fb68c11c2cc77bc0aa4/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:06.985466 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a26a7-b5a3-48c9-9cb7-cb1d88bfffab None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:06.985751 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a26a7-b5a3-48c9-9cb7-cb1d88bfffab None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:06.986027 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a26a7-b5a3-48c9-9cb7-cb1d88bfffab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:07.015977 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-c45a26a7-b5a3-48c9-9cb7-cb1d88bfffab None None] MFA Rules not processed for user `c506636f5f984fb68c11c2cc77bc0aa4`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:58:07.017192 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-c45a26a7-b5a3-48c9-9cb7-cb1d88bfffab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:07.086800 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1327/2671] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:58:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:58:07.088073 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d5f39d-ee6a-4054-833b-d7c16c4d3ee5 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:07.099393 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:07.168721 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d5f39d-ee6a-4054-833b-d7c16c4d3ee5 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:07.175580 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de9f5a8-0fdd-47b2-8d5b-012e6fa52d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:07.179385 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9f5a8-0fdd-47b2-8d5b-012e6fa52d3e admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:07.179385 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9f5a8-0fdd-47b2-8d5b-012e6fa52d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:07.179385 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9f5a8-0fdd-47b2-8d5b-012e6fa52d3e admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:07.179385 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de9f5a8-0fdd-47b2-8d5b-012e6fa52d3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:07.179385 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de9f5a8-0fdd-47b2-8d5b-012e6fa52d3e admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:07.305182 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1345/2672] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:58:07 2022] POST /identity/v3/projects => generated 371 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:58:07.306644 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45a26a7-b5a3-48c9-9cb7-cb1d88bfffab None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:07.317469 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45a26a7-b5a3-48c9-9cb7-cb1d88bfffab None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:07.353688 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:07.384042 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb6b16-1181-4ca5-ad14-0c1d926e7bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:07.384677 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb6b16-1181-4ca5-ad14-0c1d926e7bf5 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:07.385024 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb6b16-1181-4ca5-ad14-0c1d926e7bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:07.385363 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb6b16-1181-4ca5-ad14-0c1d926e7bf5 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:07.386054 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fb6b16-1181-4ca5-ad14-0c1d926e7bf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:07.388206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fb6b16-1181-4ca5-ad14-0c1d926e7bf5 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:07.542702 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1328/2673] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:58:07 2022] POST /identity/v3/users => generated 292 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:58:07.542927 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de9f5a8-0fdd-47b2-8d5b-012e6fa52d3e admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:07.554543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de9f5a8-0fdd-47b2-8d5b-012e6fa52d3e admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:07.559451 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e4e3f-260d-48f5-9db5-0d82ca4185ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:07.560071 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e4e3f-260d-48f5-9db5-0d82ca4185ae admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:07.560508 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e4e3f-260d-48f5-9db5-0d82ca4185ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:07.560791 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e4e3f-260d-48f5-9db5-0d82ca4185ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:07.565685 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11e4e3f-260d-48f5-9db5-0d82ca4185ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:07.567500 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11e4e3f-260d-48f5-9db5-0d82ca4185ae admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:07.581077 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1346/2674] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:58:07.585208 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb6b16-1181-4ca5-ad14-0c1d926e7bf5 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:07.599707 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb6b16-1181-4ca5-ad14-0c1d926e7bf5 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:07.604903 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3167dbde-50b2-4dac-9515-0518e28d06b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:07.605587 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167dbde-50b2-4dac-9515-0518e28d06b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:07.605868 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167dbde-50b2-4dac-9515-0518e28d06b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:07.606145 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167dbde-50b2-4dac-9515-0518e28d06b9 admin admin] PATH_INFO: `/v3/projects/5665ea2d372b4c53abb06bb98821ffa2/users/581d854c543e4f99a9753e66f9e6feb7/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:07.628409 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3167dbde-50b2-4dac-9515-0518e28d06b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5665ea2d372b4c53abb06bb98821ffa2, user_id=581d854c543e4f99a9753e66f9e6feb7, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:07.633486 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3167dbde-50b2-4dac-9515-0518e28d06b9 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:07.732158 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1329/2675] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:58:07 2022] PUT /identity/v3/projects/5665ea2d372b4c53abb06bb98821ffa2/users/581d854c543e4f99a9753e66f9e6feb7/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:07.736802 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e4e3f-260d-48f5-9db5-0d82ca4185ae admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:07.756127 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e4e3f-260d-48f5-9db5-0d82ca4185ae admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:07.837175 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b4d68-5872-4fd2-a771-534ec76c9986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:07.838234 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b4d68-5872-4fd2-a771-534ec76c9986 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:07.838737 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b4d68-5872-4fd2-a771-534ec76c9986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:07.839168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b4d68-5872-4fd2-a771-534ec76c9986 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:07.839911 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275b4d68-5872-4fd2-a771-534ec76c9986 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:07.842584 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275b4d68-5872-4fd2-a771-534ec76c9986 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:07.855078 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1347/2676] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:58:07.859792 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3167dbde-50b2-4dac-9515-0518e28d06b9 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:07.870463 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3167dbde-50b2-4dac-9515-0518e28d06b9 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:07.876458 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e31a4b-1a1c-4a35-9ddb-28fc40e7f1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:07.876638 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e31a4b-1a1c-4a35-9ddb-28fc40e7f1b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:07.876770 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e31a4b-1a1c-4a35-9ddb-28fc40e7f1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:07.877025 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e31a4b-1a1c-4a35-9ddb-28fc40e7f1b2 admin admin] PATH_INFO: `/v3/projects/5665ea2d372b4c53abb06bb98821ffa2/users/581d854c543e4f99a9753e66f9e6feb7/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:07.879877 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e31a4b-1a1c-4a35-9ddb-28fc40e7f1b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5665ea2d372b4c53abb06bb98821ffa2, user_id=581d854c543e4f99a9753e66f9e6feb7, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:07.882198 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e31a4b-1a1c-4a35-9ddb-28fc40e7f1b2 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:07.893906 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1330/2677] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:58:07 2022] PUT /identity/v3/projects/5665ea2d372b4c53abb06bb98821ffa2/users/581d854c543e4f99a9753e66f9e6feb7/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:58:07.904623 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76759029-aff6-4020-b331-15cc196c2d85 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:07.904907 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76759029-aff6-4020-b331-15cc196c2d85 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:07.905482 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76759029-aff6-4020-b331-15cc196c2d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:07.937158 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-76759029-aff6-4020-b331-15cc196c2d85 None None] MFA Rules not processed for user `581d854c543e4f99a9753e66f9e6feb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:58:07.939018 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-76759029-aff6-4020-b331-15cc196c2d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:08.010429 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1348/2678] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:58:07 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:58:08.022974 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e31a4b-1a1c-4a35-9ddb-28fc40e7f1b2 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:08.026059 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-f3e31a4b-1a1c-4a35-9ddb-28fc40e7f1b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:08.129540 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e31a4b-1a1c-4a35-9ddb-28fc40e7f1b2 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:08.134713 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d837ce6-2091-4199-9aaa-55e972dac56e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:08.135718 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d837ce6-2091-4199-9aaa-55e972dac56e service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:08.136109 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d837ce6-2091-4199-9aaa-55e972dac56e service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:08.136493 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d837ce6-2091-4199-9aaa-55e972dac56e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:08.152495 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d837ce6-2091-4199-9aaa-55e972dac56e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:08.155792 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d837ce6-2091-4199-9aaa-55e972dac56e service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:08.173419 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1331/2679] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:58:08 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:58:09.215041 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76759029-aff6-4020-b331-15cc196c2d85 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:09.221246 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:09.289010 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76759029-aff6-4020-b331-15cc196c2d85 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:09.299153 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec13b77-a76a-4968-9b56-a687bfea7311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:09.300286 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec13b77-a76a-4968-9b56-a687bfea7311 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:09.300979 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec13b77-a76a-4968-9b56-a687bfea7311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:09.301468 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec13b77-a76a-4968-9b56-a687bfea7311 admin admin] PATH_INFO: `/v3/users/c506636f5f984fb68c11c2cc77bc0aa4` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:09.303409 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec13b77-a76a-4968-9b56-a687bfea7311 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c506636f5f984fb68c11c2cc77bc0aa4)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:09.305463 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec13b77-a76a-4968-9b56-a687bfea7311 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:09.546586 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-1ec13b77-a76a-4968-9b56-a687bfea7311 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c506636f5f984fb68c11c2cc77bc0aa4'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:09.561386 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-1ec13b77-a76a-4968-9b56-a687bfea7311 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c506636f5f984fb68c11c2cc77bc0aa4'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:09.594777 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-1ec13b77-a76a-4968-9b56-a687bfea7311 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c506636f5f984fb68c11c2cc77bc0aa4'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:09.596512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-1ec13b77-a76a-4968-9b56-a687bfea7311 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c506636f5f984fb68c11c2cc77bc0aa4'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:09.688516 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-1ec13b77-a76a-4968-9b56-a687bfea7311 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c506636f5f984fb68c11c2cc77bc0aa4'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:09.698770 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1349/2680] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:58:09 2022] DELETE /identity/v3/users/c506636f5f984fb68c11c2cc77bc0aa4 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:09.701221 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca88f9-5239-44d8-9ab8-4aefec58c6d3 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:09.702038 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca88f9-5239-44d8-9ab8-4aefec58c6d3 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:09.702491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca88f9-5239-44d8-9ab8-4aefec58c6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:09.740827 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-b7ca88f9-5239-44d8-9ab8-4aefec58c6d3 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:58:09.744058 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b7ca88f9-5239-44d8-9ab8-4aefec58c6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:09.892001 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1332/2681] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:58:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:58:09.914862 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec13b77-a76a-4968-9b56-a687bfea7311 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:09.916462 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-1ec13b77-a76a-4968-9b56-a687bfea7311 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:10.004228 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec13b77-a76a-4968-9b56-a687bfea7311 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:10.008532 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f851e4-d4eb-4a72-875c-059738d9051b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:10.009225 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f851e4-d4eb-4a72-875c-059738d9051b service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:10.009649 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f851e4-d4eb-4a72-875c-059738d9051b service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:10.009997 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f851e4-d4eb-4a72-875c-059738d9051b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:10.023944 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f851e4-d4eb-4a72-875c-059738d9051b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:10.025303 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f851e4-d4eb-4a72-875c-059738d9051b service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:10.041232 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1350/2682] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:58:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:58:10.793277 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ca88f9-5239-44d8-9ab8-4aefec58c6d3 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:10.794940 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b7ca88f9-5239-44d8-9ab8-4aefec58c6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:10.814153 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ca88f9-5239-44d8-9ab8-4aefec58c6d3 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:10.819453 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61779eba-7947-44ad-bc2c-d6b0835be77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:10.820174 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61779eba-7947-44ad-bc2c-d6b0835be77d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:10.820548 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61779eba-7947-44ad-bc2c-d6b0835be77d admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:10.820884 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61779eba-7947-44ad-bc2c-d6b0835be77d admin admin] PATH_INFO: `/v3/projects/be44ac233d084dbf87abe5e0f15bb020` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:10.822102 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61779eba-7947-44ad-bc2c-d6b0835be77d admin admin] RBAC: Authorizing `identity:delete_project(project_id=be44ac233d084dbf87abe5e0f15bb020)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:10.824394 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61779eba-7947-44ad-bc2c-d6b0835be77d admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:10.836136 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:10.969188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-61779eba-7947-44ad-bc2c-d6b0835be77d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be44ac233d084dbf87abe5e0f15bb020', 'request_id': 'req-61779eba-7947-44ad-bc2c-d6b0835be77d'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:10.974715 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-61779eba-7947-44ad-bc2c-d6b0835be77d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be44ac233d084dbf87abe5e0f15bb020', 'request_id': 'req-61779eba-7947-44ad-bc2c-d6b0835be77d'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:11.042054 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-61779eba-7947-44ad-bc2c-d6b0835be77d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be44ac233d084dbf87abe5e0f15bb020', 'request_id': 'req-61779eba-7947-44ad-bc2c-d6b0835be77d'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:11.053949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-61779eba-7947-44ad-bc2c-d6b0835be77d admin admin] The token cache is being invalidate because project be44ac233d084dbf87abe5e0f15bb020 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:58:11.054242 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-61779eba-7947-44ad-bc2c-d6b0835be77d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:11.054865 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-61779eba-7947-44ad-bc2c-d6b0835be77d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:11.056619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1333/2683] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:58:10 2022] DELETE /identity/v3/projects/be44ac233d084dbf87abe5e0f15bb020 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:11.061070 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f851e4-d4eb-4a72-875c-059738d9051b service neutron] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:11.071239 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f851e4-d4eb-4a72-875c-059738d9051b service neutron] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:11.095970 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:11.130208 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e262919-1062-43e0-a5c0-e37be3bffe81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:11.130873 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e262919-1062-43e0-a5c0-e37be3bffe81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:11.131187 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e262919-1062-43e0-a5c0-e37be3bffe81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:11.131509 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e262919-1062-43e0-a5c0-e37be3bffe81 admin admin] PATH_INFO: `/v3/users/581d854c543e4f99a9753e66f9e6feb7` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:11.132724 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e262919-1062-43e0-a5c0-e37be3bffe81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=581d854c543e4f99a9753e66f9e6feb7)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:11.134659 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e262919-1062-43e0-a5c0-e37be3bffe81 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:11.272944 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-6e262919-1062-43e0-a5c0-e37be3bffe81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '581d854c543e4f99a9753e66f9e6feb7'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:11.282597 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-6e262919-1062-43e0-a5c0-e37be3bffe81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '581d854c543e4f99a9753e66f9e6feb7'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:11.297036 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-6e262919-1062-43e0-a5c0-e37be3bffe81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '581d854c543e4f99a9753e66f9e6feb7'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:11.297934 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-6e262919-1062-43e0-a5c0-e37be3bffe81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '581d854c543e4f99a9753e66f9e6feb7'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:11.361743 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-6e262919-1062-43e0-a5c0-e37be3bffe81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '581d854c543e4f99a9753e66f9e6feb7'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:11.370780 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1351/2684] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:58:11 2022] DELETE /identity/v3/users/581d854c543e4f99a9753e66f9e6feb7 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:11.877844 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61779eba-7947-44ad-bc2c-d6b0835be77d admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:11.879297 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-61779eba-7947-44ad-bc2c-d6b0835be77d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:11.893135 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61779eba-7947-44ad-bc2c-d6b0835be77d admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:11.979606 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d035c4a-a4c5-4b29-b494-cd5aaead0eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:11.982152 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d035c4a-a4c5-4b29-b494-cd5aaead0eed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:11.982152 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d035c4a-a4c5-4b29-b494-cd5aaead0eed admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:11.982152 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d035c4a-a4c5-4b29-b494-cd5aaead0eed admin admin] PATH_INFO: `/v3/projects/5665ea2d372b4c53abb06bb98821ffa2` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:11.982832 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d035c4a-a4c5-4b29-b494-cd5aaead0eed admin admin] RBAC: Authorizing `identity:delete_project(project_id=5665ea2d372b4c53abb06bb98821ffa2)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:11.984777 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d035c4a-a4c5-4b29-b494-cd5aaead0eed admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:12.099417 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-2d035c4a-a4c5-4b29-b494-cd5aaead0eed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5665ea2d372b4c53abb06bb98821ffa2', 'request_id': 'req-2d035c4a-a4c5-4b29-b494-cd5aaead0eed'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:12.106101 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-2d035c4a-a4c5-4b29-b494-cd5aaead0eed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5665ea2d372b4c53abb06bb98821ffa2', 'request_id': 'req-2d035c4a-a4c5-4b29-b494-cd5aaead0eed'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:12.161813 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-2d035c4a-a4c5-4b29-b494-cd5aaead0eed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5665ea2d372b4c53abb06bb98821ffa2', 'request_id': 'req-2d035c4a-a4c5-4b29-b494-cd5aaead0eed'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:12.174290 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-2d035c4a-a4c5-4b29-b494-cd5aaead0eed admin admin] The token cache is being invalidate because project 5665ea2d372b4c53abb06bb98821ffa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:58:12.174655 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-2d035c4a-a4c5-4b29-b494-cd5aaead0eed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:12.182229 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-2d035c4a-a4c5-4b29-b494-cd5aaead0eed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:12.182229 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1334/2685] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:58:11 2022] DELETE /identity/v3/projects/5665ea2d372b4c53abb06bb98821ffa2 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:12.182454 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6f807-7368-42eb-bd6c-33413e70d8cd None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:12.182454 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6f807-7368-42eb-bd6c-33413e70d8cd None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:12.183021 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6f807-7368-42eb-bd6c-33413e70d8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:12.202863 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-aeb6f807-7368-42eb-bd6c-33413e70d8cd None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:58:12.204901 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-aeb6f807-7368-42eb-bd6c-33413e70d8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:12.271393 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1352/2686] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:58:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:58:12.281481 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2728f2-9bf4-4e73-a51d-58fe4c05873b None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:12.281855 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2728f2-9bf4-4e73-a51d-58fe4c05873b None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:12.282196 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2728f2-9bf4-4e73-a51d-58fe4c05873b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:12.302124 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-4b2728f2-9bf4-4e73-a51d-58fe4c05873b None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:58:12.303392 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4b2728f2-9bf4-4e73-a51d-58fe4c05873b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:12.311494 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1335/2687] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:58:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:58:12.315882 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb6f807-7368-42eb-bd6c-33413e70d8cd None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:12.326319 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb6f807-7368-42eb-bd6c-33413e70d8cd None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:12.330831 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b912d-796e-4fd1-96c3-82fd1a4a21ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:12.331493 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b912d-796e-4fd1-96c3-82fd1a4a21ee admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:12.331780 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b912d-796e-4fd1-96c3-82fd1a4a21ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:12.332058 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b912d-796e-4fd1-96c3-82fd1a4a21ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:12.332700 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106b912d-796e-4fd1-96c3-82fd1a4a21ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:58:12.333005 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106b912d-796e-4fd1-96c3-82fd1a4a21ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:12.334655 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106b912d-796e-4fd1-96c3-82fd1a4a21ee admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:12.351707 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1353/2688] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:58:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:58:12.353532 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2728f2-9bf4-4e73-a51d-58fe4c05873b None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:12.362895 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2728f2-9bf4-4e73-a51d-58fe4c05873b None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:12.367867 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3ddb66-3ec7-467a-aff5-1f7a53e6fce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:12.368476 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ddb66-3ec7-467a-aff5-1f7a53e6fce6 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:12.368754 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ddb66-3ec7-467a-aff5-1f7a53e6fce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:12.369025 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ddb66-3ec7-467a-aff5-1f7a53e6fce6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:12.369658 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3ddb66-3ec7-467a-aff5-1f7a53e6fce6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:12.371578 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3ddb66-3ec7-467a-aff5-1f7a53e6fce6 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:12.467800 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:12.478695 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1336/2689] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:58:12 2022] POST /identity/v3/projects => generated 399 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:58:12.479879 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b912d-796e-4fd1-96c3-82fd1a4a21ee admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:12.492835 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b912d-796e-4fd1-96c3-82fd1a4a21ee admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:12.529758 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:12.559032 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b883ccd9-ae6f-4827-bf3d-1033fad3a4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:12.560039 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b883ccd9-ae6f-4827-bf3d-1033fad3a4bc admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:12.560392 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b883ccd9-ae6f-4827-bf3d-1033fad3a4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:12.560740 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b883ccd9-ae6f-4827-bf3d-1033fad3a4bc admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:12.561458 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b883ccd9-ae6f-4827-bf3d-1033fad3a4bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:12.563571 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b883ccd9-ae6f-4827-bf3d-1033fad3a4bc admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:12.675535 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1354/2690] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:58:12 2022] POST /identity/v3/users => generated 300 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:58:12.676866 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3ddb66-3ec7-467a-aff5-1f7a53e6fce6 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:12.688123 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3ddb66-3ec7-467a-aff5-1f7a53e6fce6 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:12.693081 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312cf7f-0565-47c6-9483-55012b44dc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:12.693714 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312cf7f-0565-47c6-9483-55012b44dc7f admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:12.694047 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312cf7f-0565-47c6-9483-55012b44dc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:12.694325 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312cf7f-0565-47c6-9483-55012b44dc7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:12.694967 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d312cf7f-0565-47c6-9483-55012b44dc7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:12.696743 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d312cf7f-0565-47c6-9483-55012b44dc7f admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:12.707937 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1337/2691] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:58:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 12:58:12.712044 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b883ccd9-ae6f-4827-bf3d-1033fad3a4bc admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:12.723255 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b883ccd9-ae6f-4827-bf3d-1033fad3a4bc admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:12.728630 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b28692e-f9df-4c7c-bcb0-20a40c63eab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:12.729312 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28692e-f9df-4c7c-bcb0-20a40c63eab3 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:12.729732 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28692e-f9df-4c7c-bcb0-20a40c63eab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:12.730122 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28692e-f9df-4c7c-bcb0-20a40c63eab3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:12.730791 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b28692e-f9df-4c7c-bcb0-20a40c63eab3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:12.733269 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b28692e-f9df-4c7c-bcb0-20a40c63eab3 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:12.744603 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1355/2692] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:58:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 12:58:12.749346 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312cf7f-0565-47c6-9483-55012b44dc7f admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:12.759742 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312cf7f-0565-47c6-9483-55012b44dc7f admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:12.764960 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9fcdc8-aa71-41f1-bbc5-d9b11f4e2647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:12.765630 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9fcdc8-aa71-41f1-bbc5-d9b11f4e2647 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:12.766009 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9fcdc8-aa71-41f1-bbc5-d9b11f4e2647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:12.766353 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9fcdc8-aa71-41f1-bbc5-d9b11f4e2647 admin admin] PATH_INFO: `/v3/projects/951d5e57e47e424e8984888b9bfc5293/users/1983a9063259432db700c5e57963d7c8/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:12.782194 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9fcdc8-aa71-41f1-bbc5-d9b11f4e2647 admin admin] RBAC: Authorizing `identity:create_grant(project_id=951d5e57e47e424e8984888b9bfc5293, user_id=1983a9063259432db700c5e57963d7c8, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:12.786143 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9fcdc8-aa71-41f1-bbc5-d9b11f4e2647 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:12.854238 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1338/2693] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:58:12 2022] PUT /identity/v3/projects/951d5e57e47e424e8984888b9bfc5293/users/1983a9063259432db700c5e57963d7c8/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:12.872893 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5de0f0-c7fb-46fa-82a3-f5533426841e None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:12.873395 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5de0f0-c7fb-46fa-82a3-f5533426841e None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:12.873901 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5de0f0-c7fb-46fa-82a3-f5533426841e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:12.912763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-6b5de0f0-c7fb-46fa-82a3-f5533426841e None None] MFA Rules not processed for user `1983a9063259432db700c5e57963d7c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:58:12.914873 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-6b5de0f0-c7fb-46fa-82a3-f5533426841e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:12.996024 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1356/2694] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:58:12 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:58:13.001398 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9fcdc8-aa71-41f1-bbc5-d9b11f4e2647 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:13.065439 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9fcdc8-aa71-41f1-bbc5-d9b11f4e2647 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:13.070070 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35144af9-4f8a-4cea-abda-7ca54e8b0daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:13.070678 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35144af9-4f8a-4cea-abda-7ca54e8b0daa admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:13.070954 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35144af9-4f8a-4cea-abda-7ca54e8b0daa admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:13.071226 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35144af9-4f8a-4cea-abda-7ca54e8b0daa admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:13.071888 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35144af9-4f8a-4cea-abda-7ca54e8b0daa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:13.073770 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35144af9-4f8a-4cea-abda-7ca54e8b0daa admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:13.178423 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1339/2695] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:58:13 2022] POST /identity/v3/projects => generated 399 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:58:13.179746 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5de0f0-c7fb-46fa-82a3-f5533426841e None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:13.190833 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5de0f0-c7fb-46fa-82a3-f5533426841e None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:13.262623 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1780bac2-f521-48c7-b54d-27620ef91723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:13.262843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780bac2-f521-48c7-b54d-27620ef91723 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:13.263114 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780bac2-f521-48c7-b54d-27620ef91723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:13.263343 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780bac2-f521-48c7-b54d-27620ef91723 admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:13.264097 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1780bac2-f521-48c7-b54d-27620ef91723 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:13.270387 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1780bac2-f521-48c7-b54d-27620ef91723 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:13.361487 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1357/2696] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:58:13 2022] POST /identity/v3/users => generated 306 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:58:13.362278 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35144af9-4f8a-4cea-abda-7ca54e8b0daa admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:13.374317 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35144af9-4f8a-4cea-abda-7ca54e8b0daa admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:13.379574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc1115-5848-4dca-be62-9a956b9514aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:13.380186 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc1115-5848-4dca-be62-9a956b9514aa admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:13.380464 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc1115-5848-4dca-be62-9a956b9514aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:13.380723 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc1115-5848-4dca-be62-9a956b9514aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:13.381382 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbc1115-5848-4dca-be62-9a956b9514aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:13.384023 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbc1115-5848-4dca-be62-9a956b9514aa admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:13.404241 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1340/2697] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:58:13 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:58:13.408411 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1780bac2-f521-48c7-b54d-27620ef91723 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:13.418832 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1780bac2-f521-48c7-b54d-27620ef91723 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:13.423508 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd43000-8c2d-4cbe-99ef-162fcc8f7a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:13.424159 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd43000-8c2d-4cbe-99ef-162fcc8f7a4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:13.424440 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd43000-8c2d-4cbe-99ef-162fcc8f7a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:13.424714 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd43000-8c2d-4cbe-99ef-162fcc8f7a4f admin admin] PATH_INFO: `/v3/projects/a33102207c5044028bdb1d50a1b80aa9/users/b0027ab00d99483ea0a4aeac8d2d2c0c/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:13.441646 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd43000-8c2d-4cbe-99ef-162fcc8f7a4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a33102207c5044028bdb1d50a1b80aa9, user_id=b0027ab00d99483ea0a4aeac8d2d2c0c, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:13.444653 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd43000-8c2d-4cbe-99ef-162fcc8f7a4f admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:13.528717 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1358/2698] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:58:13 2022] PUT /identity/v3/projects/a33102207c5044028bdb1d50a1b80aa9/users/b0027ab00d99483ea0a4aeac8d2d2c0c/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:58:13.532631 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc1115-5848-4dca-be62-9a956b9514aa admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:13.542571 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc1115-5848-4dca-be62-9a956b9514aa admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:13.604439 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f17f45e-5ada-4b6c-9b28-98887b4fce3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:13.605055 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f17f45e-5ada-4b6c-9b28-98887b4fce3c admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:13.605333 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f17f45e-5ada-4b6c-9b28-98887b4fce3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:13.605608 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f17f45e-5ada-4b6c-9b28-98887b4fce3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:13.606797 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f17f45e-5ada-4b6c-9b28-98887b4fce3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:13.608517 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f17f45e-5ada-4b6c-9b28-98887b4fce3c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:13.618873 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1341/2699] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:58:13 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 12:58:13.624209 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd43000-8c2d-4cbe-99ef-162fcc8f7a4f admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:13.635629 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd43000-8c2d-4cbe-99ef-162fcc8f7a4f admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:13.640865 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00c83ff-2076-4192-830c-4333394c8c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:13.641480 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c83ff-2076-4192-830c-4333394c8c3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:13.642088 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c83ff-2076-4192-830c-4333394c8c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:13.642088 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c83ff-2076-4192-830c-4333394c8c3e admin admin] PATH_INFO: `/v3/projects/a33102207c5044028bdb1d50a1b80aa9/users/b0027ab00d99483ea0a4aeac8d2d2c0c/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:13.644085 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00c83ff-2076-4192-830c-4333394c8c3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a33102207c5044028bdb1d50a1b80aa9, user_id=b0027ab00d99483ea0a4aeac8d2d2c0c, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:13.647122 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00c83ff-2076-4192-830c-4333394c8c3e admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:13.649159 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:13.659045 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1359/2700] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:58:13 2022] PUT /identity/v3/projects/a33102207c5044028bdb1d50a1b80aa9/users/b0027ab00d99483ea0a4aeac8d2d2c0c/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:13.670216 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab97d0-7a4a-48c5-bf9c-029f87e11887 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:13.670540 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab97d0-7a4a-48c5-bf9c-029f87e11887 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:13.670968 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab97d0-7a4a-48c5-bf9c-029f87e11887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:13.709060 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-beab97d0-7a4a-48c5-bf9c-029f87e11887 None None] MFA Rules not processed for user `b0027ab00d99483ea0a4aeac8d2d2c0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:58:13.710427 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-beab97d0-7a4a-48c5-bf9c-029f87e11887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:13.740284 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:13.788812 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1342/2701] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:58:13 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:58:13.814545 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00c83ff-2076-4192-830c-4333394c8c3e admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:13.817170 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-d00c83ff-2076-4192-830c-4333394c8c3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:13.908898 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00c83ff-2076-4192-830c-4333394c8c3e admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:13.913467 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547a45cb-20fe-4a5e-aa62-6ad014389d5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:13.914091 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a45cb-20fe-4a5e-aa62-6ad014389d5b service cinder] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:13.914376 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a45cb-20fe-4a5e-aa62-6ad014389d5b service cinder] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:13.914653 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a45cb-20fe-4a5e-aa62-6ad014389d5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:13.931147 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547a45cb-20fe-4a5e-aa62-6ad014389d5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:13.935392 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547a45cb-20fe-4a5e-aa62-6ad014389d5b service cinder] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:13.946355 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1360/2702] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:58:13 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:58:14.165904 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab97d0-7a4a-48c5-bf9c-029f87e11887 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:14.177788 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab97d0-7a4a-48c5-bf9c-029f87e11887 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:14.184074 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fecacc9-7f1f-4e29-a6bf-bed57141f95f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:14.184814 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fecacc9-7f1f-4e29-a6bf-bed57141f95f service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:14.185516 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fecacc9-7f1f-4e29-a6bf-bed57141f95f service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:14.185863 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fecacc9-7f1f-4e29-a6bf-bed57141f95f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:14.279840 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fecacc9-7f1f-4e29-a6bf-bed57141f95f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:14.282107 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fecacc9-7f1f-4e29-a6bf-bed57141f95f service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:14.292759 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1343/2703] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:58:14 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:58:14.525170 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547a45cb-20fe-4a5e-aa62-6ad014389d5b service cinder] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:14.526686 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-547a45cb-20fe-4a5e-aa62-6ad014389d5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:14.597293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547a45cb-20fe-4a5e-aa62-6ad014389d5b service cinder] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:14.602157 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a7d737-30dc-47cd-a67e-835077419063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:14.602799 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7d737-30dc-47cd-a67e-835077419063 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:14.603077 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7d737-30dc-47cd-a67e-835077419063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:14.603383 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7d737-30dc-47cd-a67e-835077419063 admin admin] PATH_INFO: `/v3/users/bafb4a06c49d47cdb970e3f9e08a2543` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:14.604530 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a7d737-30dc-47cd-a67e-835077419063 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bafb4a06c49d47cdb970e3f9e08a2543)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:14.606875 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a7d737-30dc-47cd-a67e-835077419063 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:14.857116 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-41a7d737-30dc-47cd-a67e-835077419063 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bafb4a06c49d47cdb970e3f9e08a2543'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:14.864340 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-41a7d737-30dc-47cd-a67e-835077419063 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bafb4a06c49d47cdb970e3f9e08a2543'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:14.877636 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-41a7d737-30dc-47cd-a67e-835077419063 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bafb4a06c49d47cdb970e3f9e08a2543'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:14.878338 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-41a7d737-30dc-47cd-a67e-835077419063 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bafb4a06c49d47cdb970e3f9e08a2543'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:14.968163 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-41a7d737-30dc-47cd-a67e-835077419063 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bafb4a06c49d47cdb970e3f9e08a2543'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:14.974574 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1361/2704] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:58:14 2022] DELETE /identity/v3/users/bafb4a06c49d47cdb970e3f9e08a2543 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:15.318898 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fecacc9-7f1f-4e29-a6bf-bed57141f95f service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:15.336630 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fecacc9-7f1f-4e29-a6bf-bed57141f95f service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:15.402884 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac60ff1-651d-4de7-b67a-91064b3cfb5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:15.403656 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac60ff1-651d-4de7-b67a-91064b3cfb5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:15.403939 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac60ff1-651d-4de7-b67a-91064b3cfb5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:15.404216 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac60ff1-651d-4de7-b67a-91064b3cfb5a admin admin] PATH_INFO: `/v3/projects/06a284bf10ff47d6bc58de80f92c164a` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:15.405393 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac60ff1-651d-4de7-b67a-91064b3cfb5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=06a284bf10ff47d6bc58de80f92c164a)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:15.407408 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac60ff1-651d-4de7-b67a-91064b3cfb5a admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:15.533693 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-8ac60ff1-651d-4de7-b67a-91064b3cfb5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06a284bf10ff47d6bc58de80f92c164a', 'request_id': 'req-8ac60ff1-651d-4de7-b67a-91064b3cfb5a'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:15.539196 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-8ac60ff1-651d-4de7-b67a-91064b3cfb5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06a284bf10ff47d6bc58de80f92c164a', 'request_id': 'req-8ac60ff1-651d-4de7-b67a-91064b3cfb5a'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:15.540899 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:15.642799 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-8ac60ff1-651d-4de7-b67a-91064b3cfb5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06a284bf10ff47d6bc58de80f92c164a', 'request_id': 'req-8ac60ff1-651d-4de7-b67a-91064b3cfb5a'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:15.654939 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-8ac60ff1-651d-4de7-b67a-91064b3cfb5a admin admin] The token cache is being invalidate because project 06a284bf10ff47d6bc58de80f92c164a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:58:15.655230 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-8ac60ff1-651d-4de7-b67a-91064b3cfb5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:15.655901 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-8ac60ff1-651d-4de7-b67a-91064b3cfb5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:15.657381 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1344/2705] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:58:15 2022] DELETE /identity/v3/projects/06a284bf10ff47d6bc58de80f92c164a => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:15.661476 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a7d737-30dc-47cd-a67e-835077419063 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:15.662924 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-41a7d737-30dc-47cd-a67e-835077419063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:15.676016 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a7d737-30dc-47cd-a67e-835077419063 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:15.688438 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:15.732896 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ddc98-86e3-4030-b8c2-39f61be03332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:15.733544 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ddc98-86e3-4030-b8c2-39f61be03332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:15.734018 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ddc98-86e3-4030-b8c2-39f61be03332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:15.734702 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ddc98-86e3-4030-b8c2-39f61be03332 admin admin] PATH_INFO: `/v3/users/5347c3430eb440f7916fbbeee37c1048` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:15.735912 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6ddc98-86e3-4030-b8c2-39f61be03332 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5347c3430eb440f7916fbbeee37c1048)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:15.737847 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6ddc98-86e3-4030-b8c2-39f61be03332 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:15.867526 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-dd6ddc98-86e3-4030-b8c2-39f61be03332 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5347c3430eb440f7916fbbeee37c1048'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:15.875406 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-dd6ddc98-86e3-4030-b8c2-39f61be03332 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5347c3430eb440f7916fbbeee37c1048'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:15.887957 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-dd6ddc98-86e3-4030-b8c2-39f61be03332 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5347c3430eb440f7916fbbeee37c1048'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:15.888671 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-dd6ddc98-86e3-4030-b8c2-39f61be03332 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5347c3430eb440f7916fbbeee37c1048'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:15.968316 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-dd6ddc98-86e3-4030-b8c2-39f61be03332 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5347c3430eb440f7916fbbeee37c1048'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:15.974901 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1362/2706] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:58:15 2022] DELETE /identity/v3/users/5347c3430eb440f7916fbbeee37c1048 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:16.208881 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac60ff1-651d-4de7-b67a-91064b3cfb5a admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:16.210356 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-8ac60ff1-651d-4de7-b67a-91064b3cfb5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:16.224662 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac60ff1-651d-4de7-b67a-91064b3cfb5a admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:16.288725 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc85672-dc38-4894-9d3a-016bb2192ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:16.289444 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc85672-dc38-4894-9d3a-016bb2192ea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:16.289793 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc85672-dc38-4894-9d3a-016bb2192ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:16.290132 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc85672-dc38-4894-9d3a-016bb2192ea6 admin admin] PATH_INFO: `/v3/projects/6ca39e772a0145148115322e8c815c59` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:16.291525 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc85672-dc38-4894-9d3a-016bb2192ea6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ca39e772a0145148115322e8c815c59)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:16.293606 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc85672-dc38-4894-9d3a-016bb2192ea6 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:16.448366 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-acc85672-dc38-4894-9d3a-016bb2192ea6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ca39e772a0145148115322e8c815c59', 'request_id': 'req-acc85672-dc38-4894-9d3a-016bb2192ea6'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:16.454534 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-acc85672-dc38-4894-9d3a-016bb2192ea6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ca39e772a0145148115322e8c815c59', 'request_id': 'req-acc85672-dc38-4894-9d3a-016bb2192ea6'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:16.539471 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-acc85672-dc38-4894-9d3a-016bb2192ea6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ca39e772a0145148115322e8c815c59', 'request_id': 'req-acc85672-dc38-4894-9d3a-016bb2192ea6'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:16.552453 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-acc85672-dc38-4894-9d3a-016bb2192ea6 admin admin] The token cache is being invalidate because project 6ca39e772a0145148115322e8c815c59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:58:16.552742 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-acc85672-dc38-4894-9d3a-016bb2192ea6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:16.553383 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-acc85672-dc38-4894-9d3a-016bb2192ea6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:16.554883 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1345/2707] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:58:16 2022] DELETE /identity/v3/projects/6ca39e772a0145148115322e8c815c59 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:19.577004 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ddc98-86e3-4030-b8c2-39f61be03332 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:19.578420 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-dd6ddc98-86e3-4030-b8c2-39f61be03332 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:19.591100 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ddc98-86e3-4030-b8c2-39f61be03332 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:19.647461 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5fae2-f70c-437d-ab02-500687a77f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:19.648088 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5fae2-f70c-437d-ab02-500687a77f16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:19.648368 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5fae2-f70c-437d-ab02-500687a77f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:19.648639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5fae2-f70c-437d-ab02-500687a77f16 admin admin] PATH_INFO: `/v3/users/1983a9063259432db700c5e57963d7c8` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:19.649765 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5fae2-f70c-437d-ab02-500687a77f16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1983a9063259432db700c5e57963d7c8)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:19.651754 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5fae2-f70c-437d-ab02-500687a77f16 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:19.796163 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:19.804499 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-5ef5fae2-f70c-437d-ab02-500687a77f16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1983a9063259432db700c5e57963d7c8'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:19.811961 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-5ef5fae2-f70c-437d-ab02-500687a77f16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1983a9063259432db700c5e57963d7c8'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:19.823483 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-5ef5fae2-f70c-437d-ab02-500687a77f16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1983a9063259432db700c5e57963d7c8'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:19.824169 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-5ef5fae2-f70c-437d-ab02-500687a77f16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1983a9063259432db700c5e57963d7c8'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:19.892840 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-5ef5fae2-f70c-437d-ab02-500687a77f16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1983a9063259432db700c5e57963d7c8'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:19.898992 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1363/2708] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:58:19 2022] DELETE /identity/v3/users/1983a9063259432db700c5e57963d7c8 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:19.903911 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2179980-d45d-4fac-83b1-abbc8a79986c None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:19.906470 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2179980-d45d-4fac-83b1-abbc8a79986c None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:19.906470 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2179980-d45d-4fac-83b1-abbc8a79986c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:19.924082 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-c2179980-d45d-4fac-83b1-abbc8a79986c None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:58:19.925278 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-c2179980-d45d-4fac-83b1-abbc8a79986c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:19.983992 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1346/2709] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:58:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:58:19.993877 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5fae2-f70c-437d-ab02-500687a77f16 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:19.995280 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-5ef5fae2-f70c-437d-ab02-500687a77f16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:20.055078 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5fae2-f70c-437d-ab02-500687a77f16 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:20.058669 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34019c-6713-4162-80d9-2acd5f16b133 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:20.059688 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34019c-6713-4162-80d9-2acd5f16b133 service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:20.059979 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34019c-6713-4162-80d9-2acd5f16b133 service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:20.060250 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34019c-6713-4162-80d9-2acd5f16b133 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:20.069283 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c34019c-6713-4162-80d9-2acd5f16b133 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:20.070703 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c34019c-6713-4162-80d9-2acd5f16b133 service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:20.080466 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1364/2710] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:58:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:58:20.472494 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2179980-d45d-4fac-83b1-abbc8a79986c None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:20.477616 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-c2179980-d45d-4fac-83b1-abbc8a79986c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:20.486278 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2179980-d45d-4fac-83b1-abbc8a79986c None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:20.490857 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3bda33-0226-43a5-9b64-58e97439cc7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:20.491518 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3bda33-0226-43a5-9b64-58e97439cc7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:20.492048 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3bda33-0226-43a5-9b64-58e97439cc7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:20.492331 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3bda33-0226-43a5-9b64-58e97439cc7c admin admin] PATH_INFO: `/v3/projects/951d5e57e47e424e8984888b9bfc5293` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:20.493453 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3bda33-0226-43a5-9b64-58e97439cc7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=951d5e57e47e424e8984888b9bfc5293)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:20.495372 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3bda33-0226-43a5-9b64-58e97439cc7c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:20.507276 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:20.627589 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-da3bda33-0226-43a5-9b64-58e97439cc7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '951d5e57e47e424e8984888b9bfc5293', 'request_id': 'req-da3bda33-0226-43a5-9b64-58e97439cc7c'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:20.632992 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-da3bda33-0226-43a5-9b64-58e97439cc7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '951d5e57e47e424e8984888b9bfc5293', 'request_id': 'req-da3bda33-0226-43a5-9b64-58e97439cc7c'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:20.721690 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-da3bda33-0226-43a5-9b64-58e97439cc7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '951d5e57e47e424e8984888b9bfc5293', 'request_id': 'req-da3bda33-0226-43a5-9b64-58e97439cc7c'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:20.733804 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-da3bda33-0226-43a5-9b64-58e97439cc7c admin admin] The token cache is being invalidate because project 951d5e57e47e424e8984888b9bfc5293 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:58:20.734089 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-da3bda33-0226-43a5-9b64-58e97439cc7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:20.734701 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-da3bda33-0226-43a5-9b64-58e97439cc7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:20.736207 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1347/2711] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:58:20 2022] DELETE /identity/v3/projects/951d5e57e47e424e8984888b9bfc5293 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:58:20.740224 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34019c-6713-4162-80d9-2acd5f16b133 service neutron] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:20.751004 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34019c-6713-4162-80d9-2acd5f16b133 service neutron] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:20.806139 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5954da0-5272-43dd-b0ac-f68ba631474d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:20.806753 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5954da0-5272-43dd-b0ac-f68ba631474d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:20.807032 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5954da0-5272-43dd-b0ac-f68ba631474d admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:20.807304 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5954da0-5272-43dd-b0ac-f68ba631474d admin admin] PATH_INFO: `/v3/users/b0027ab00d99483ea0a4aeac8d2d2c0c` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:20.808437 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5954da0-5272-43dd-b0ac-f68ba631474d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0027ab00d99483ea0a4aeac8d2d2c0c)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:20.810845 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5954da0-5272-43dd-b0ac-f68ba631474d admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:21.021627 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-f5954da0-5272-43dd-b0ac-f68ba631474d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0027ab00d99483ea0a4aeac8d2d2c0c'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:21.028606 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-f5954da0-5272-43dd-b0ac-f68ba631474d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0027ab00d99483ea0a4aeac8d2d2c0c'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:21.030465 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:21.040855 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-f5954da0-5272-43dd-b0ac-f68ba631474d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0027ab00d99483ea0a4aeac8d2d2c0c'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:21.041615 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-f5954da0-5272-43dd-b0ac-f68ba631474d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0027ab00d99483ea0a4aeac8d2d2c0c'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:21.104131 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-f5954da0-5272-43dd-b0ac-f68ba631474d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0027ab00d99483ea0a4aeac8d2d2c0c'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:21.110707 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1365/2712] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:58:20 2022] DELETE /identity/v3/users/b0027ab00d99483ea0a4aeac8d2d2c0c => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:21.506417 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3bda33-0226-43a5-9b64-58e97439cc7c admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:21.507917 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-da3bda33-0226-43a5-9b64-58e97439cc7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:21.520728 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3bda33-0226-43a5-9b64-58e97439cc7c admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:21.578260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761e6b68-2779-4a49-aa89-05e0efa647ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:21.578895 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e6b68-2779-4a49-aa89-05e0efa647ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:21.579175 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e6b68-2779-4a49-aa89-05e0efa647ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:21.579481 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e6b68-2779-4a49-aa89-05e0efa647ce admin admin] PATH_INFO: `/v3/projects/a33102207c5044028bdb1d50a1b80aa9` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:21.580588 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761e6b68-2779-4a49-aa89-05e0efa647ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=a33102207c5044028bdb1d50a1b80aa9)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:21.582475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761e6b68-2779-4a49-aa89-05e0efa647ce admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:21.694678 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-761e6b68-2779-4a49-aa89-05e0efa647ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a33102207c5044028bdb1d50a1b80aa9', 'request_id': 'req-761e6b68-2779-4a49-aa89-05e0efa647ce'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:21.701068 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-761e6b68-2779-4a49-aa89-05e0efa647ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a33102207c5044028bdb1d50a1b80aa9', 'request_id': 'req-761e6b68-2779-4a49-aa89-05e0efa647ce'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:21.779099 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-761e6b68-2779-4a49-aa89-05e0efa647ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a33102207c5044028bdb1d50a1b80aa9', 'request_id': 'req-761e6b68-2779-4a49-aa89-05e0efa647ce'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:21.790825 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-761e6b68-2779-4a49-aa89-05e0efa647ce admin admin] The token cache is being invalidate because project a33102207c5044028bdb1d50a1b80aa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:58:21.791115 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-761e6b68-2779-4a49-aa89-05e0efa647ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:21.791798 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-761e6b68-2779-4a49-aa89-05e0efa647ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:21.793352 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1348/2713] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:58:21 2022] DELETE /identity/v3/projects/a33102207c5044028bdb1d50a1b80aa9 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:21.798838 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c05623-a0cc-4280-9597-c3aa015271f3 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:21.799303 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c05623-a0cc-4280-9597-c3aa015271f3 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:21.799437 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c05623-a0cc-4280-9597-c3aa015271f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:21.820879 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-46c05623-a0cc-4280-9597-c3aa015271f3 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:58:21.822147 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-46c05623-a0cc-4280-9597-c3aa015271f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:21.881051 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1366/2714] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:58:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:58:21.888915 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb17fd-da74-453b-8ae4-16b86eefd7fe None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:21.889204 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb17fd-da74-453b-8ae4-16b86eefd7fe None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:21.889487 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb17fd-da74-453b-8ae4-16b86eefd7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:21.908731 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-edfb17fd-da74-453b-8ae4-16b86eefd7fe None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:58:21.909927 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-edfb17fd-da74-453b-8ae4-16b86eefd7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:21.917741 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1349/2715] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:58:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:58:21.922717 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c05623-a0cc-4280-9597-c3aa015271f3 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:21.932425 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c05623-a0cc-4280-9597-c3aa015271f3 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:21.938068 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d1ef4-1967-4b22-bfc2-2ad3e6ebaa9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:21.938674 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d1ef4-1967-4b22-bfc2-2ad3e6ebaa9a admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:21.938985 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d1ef4-1967-4b22-bfc2-2ad3e6ebaa9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:21.939260 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d1ef4-1967-4b22-bfc2-2ad3e6ebaa9a admin admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:21.939932 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10d1ef4-1967-4b22-bfc2-2ad3e6ebaa9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:58:21.940240 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10d1ef4-1967-4b22-bfc2-2ad3e6ebaa9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:21.941821 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10d1ef4-1967-4b22-bfc2-2ad3e6ebaa9a admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:21.953767 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1367/2716] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:58:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:58:21.957852 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb17fd-da74-453b-8ae4-16b86eefd7fe None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:21.967738 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb17fd-da74-453b-8ae4-16b86eefd7fe None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:21.972433 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f86ef41-3a2c-4a05-b067-079c126d67ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:21.973035 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86ef41-3a2c-4a05-b067-079c126d67ea admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:21.973334 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86ef41-3a2c-4a05-b067-079c126d67ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:21.973593 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86ef41-3a2c-4a05-b067-079c126d67ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:21.974219 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f86ef41-3a2c-4a05-b067-079c126d67ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:21.976155 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f86ef41-3a2c-4a05-b067-079c126d67ea admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:22.088843 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1350/2717] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:58:21 2022] POST /identity/v3/projects => generated 389 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:58:22.094252 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d1ef4-1967-4b22-bfc2-2ad3e6ebaa9a admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:22.104125 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d1ef4-1967-4b22-bfc2-2ad3e6ebaa9a admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:22.159802 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2180718d-8e53-491d-a3e9-7db32091c04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:22.160399 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2180718d-8e53-491d-a3e9-7db32091c04f admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:22.160678 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2180718d-8e53-491d-a3e9-7db32091c04f admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:22.160949 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2180718d-8e53-491d-a3e9-7db32091c04f admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:22.161577 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2180718d-8e53-491d-a3e9-7db32091c04f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:22.163493 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2180718d-8e53-491d-a3e9-7db32091c04f admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:22.254490 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1368/2718] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:58:22 2022] POST /identity/v3/users => generated 295 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:58:22.258717 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f86ef41-3a2c-4a05-b067-079c126d67ea admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:22.264154 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:22.269488 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f86ef41-3a2c-4a05-b067-079c126d67ea admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:22.274008 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0792357-d75f-40a2-93e2-ec3e909e59ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:22.274597 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0792357-d75f-40a2-93e2-ec3e909e59ce admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:22.274871 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0792357-d75f-40a2-93e2-ec3e909e59ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:22.275138 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0792357-d75f-40a2-93e2-ec3e909e59ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:22.275799 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0792357-d75f-40a2-93e2-ec3e909e59ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:22.277368 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0792357-d75f-40a2-93e2-ec3e909e59ce admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:22.287603 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1351/2719] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:58:22.291526 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2180718d-8e53-491d-a3e9-7db32091c04f admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:22.302138 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2180718d-8e53-491d-a3e9-7db32091c04f admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:22.306751 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c45e57-3094-4b24-affe-1b43c5800722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:22.307366 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c45e57-3094-4b24-affe-1b43c5800722 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:22.307649 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c45e57-3094-4b24-affe-1b43c5800722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:22.307919 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c45e57-3094-4b24-affe-1b43c5800722 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:22.308550 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c45e57-3094-4b24-affe-1b43c5800722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:22.310340 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c45e57-3094-4b24-affe-1b43c5800722 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:22.320977 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1369/2720] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:58:22.325449 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0792357-d75f-40a2-93e2-ec3e909e59ce admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:22.335309 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0792357-d75f-40a2-93e2-ec3e909e59ce admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:22.340052 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ae7e9-4903-4eeb-a5c5-5051158f7b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:22.341676 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ae7e9-4903-4eeb-a5c5-5051158f7b29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:22.341972 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ae7e9-4903-4eeb-a5c5-5051158f7b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:22.342246 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ae7e9-4903-4eeb-a5c5-5051158f7b29 admin admin] PATH_INFO: `/v3/projects/807a5f053ed742e38ca2c23d1e9e577f/users/20817b3b0b2e4038a8ef17723ea7fd27/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:22.358745 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229ae7e9-4903-4eeb-a5c5-5051158f7b29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=807a5f053ed742e38ca2c23d1e9e577f, user_id=20817b3b0b2e4038a8ef17723ea7fd27, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:22.361464 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229ae7e9-4903-4eeb-a5c5-5051158f7b29 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:22.419522 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1352/2721] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:58:22 2022] PUT /identity/v3/projects/807a5f053ed742e38ca2c23d1e9e577f/users/20817b3b0b2e4038a8ef17723ea7fd27/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:22.429685 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef19662-6d06-440b-8c38-d39bba27083a None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:22.429970 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef19662-6d06-440b-8c38-d39bba27083a None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:22.430245 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef19662-6d06-440b-8c38-d39bba27083a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:22.458162 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-bef19662-6d06-440b-8c38-d39bba27083a None None] MFA Rules not processed for user `20817b3b0b2e4038a8ef17723ea7fd27`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:58:22.459389 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-bef19662-6d06-440b-8c38-d39bba27083a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:22.489063 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:22.524079 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1370/2722] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:58:22 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:58:22.529195 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ae7e9-4903-4eeb-a5c5-5051158f7b29 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:22.591742 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ae7e9-4903-4eeb-a5c5-5051158f7b29 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:22.596274 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa7c49-c113-42d6-bc5f-13bbe8342005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:22.596856 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa7c49-c113-42d6-bc5f-13bbe8342005 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:22.597133 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa7c49-c113-42d6-bc5f-13bbe8342005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:22.597402 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa7c49-c113-42d6-bc5f-13bbe8342005 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:22.598030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa7c49-c113-42d6-bc5f-13bbe8342005 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:22.599952 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa7c49-c113-42d6-bc5f-13bbe8342005 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:22.682437 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1353/2723] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:58:22 2022] POST /identity/v3/projects => generated 389 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:58:22.687755 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef19662-6d06-440b-8c38-d39bba27083a None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:22.698513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef19662-6d06-440b-8c38-d39bba27083a None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:22.753501 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35df7d1-26a9-49f9-91d7-66cb2742768e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:22.754105 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35df7d1-26a9-49f9-91d7-66cb2742768e admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:22.754399 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35df7d1-26a9-49f9-91d7-66cb2742768e admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:22.754651 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35df7d1-26a9-49f9-91d7-66cb2742768e admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:22.755278 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35df7d1-26a9-49f9-91d7-66cb2742768e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:22.757194 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35df7d1-26a9-49f9-91d7-66cb2742768e admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:22.824540 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1371/2724] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:58:22 2022] POST /identity/v3/users => generated 301 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:58:22.828622 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa7c49-c113-42d6-bc5f-13bbe8342005 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:22.838851 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa7c49-c113-42d6-bc5f-13bbe8342005 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:22.844195 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c669a346-4e2e-4592-b899-c198007addac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:22.844723 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669a346-4e2e-4592-b899-c198007addac admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:22.845000 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669a346-4e2e-4592-b899-c198007addac admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:22.845268 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669a346-4e2e-4592-b899-c198007addac admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:22.845896 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c669a346-4e2e-4592-b899-c198007addac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:22.847871 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c669a346-4e2e-4592-b899-c198007addac admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:22.859292 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1354/2725] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 12:58:22.863587 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35df7d1-26a9-49f9-91d7-66cb2742768e admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:22.875447 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35df7d1-26a9-49f9-91d7-66cb2742768e admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:22.880303 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55162e-f801-4c2a-bab1-edf8825635a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:22.881286 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55162e-f801-4c2a-bab1-edf8825635a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:22.881639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55162e-f801-4c2a-bab1-edf8825635a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:22.882069 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55162e-f801-4c2a-bab1-edf8825635a3 admin admin] PATH_INFO: `/v3/projects/7fedb037aba0465c9f3dd7e441f17cb3/users/add52b3a7d5e4992b14116d258ebbaba/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:22.897610 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa55162e-f801-4c2a-bab1-edf8825635a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fedb037aba0465c9f3dd7e441f17cb3, user_id=add52b3a7d5e4992b14116d258ebbaba, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:22.900545 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa55162e-f801-4c2a-bab1-edf8825635a3 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:22.956591 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1372/2726] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:58:22 2022] PUT /identity/v3/projects/7fedb037aba0465c9f3dd7e441f17cb3/users/add52b3a7d5e4992b14116d258ebbaba/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:22.960555 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c669a346-4e2e-4592-b899-c198007addac admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:22.971856 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c669a346-4e2e-4592-b899-c198007addac admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:23.026979 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a6a54-4fe3-4969-a9e3-404a37d4122d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:23.027624 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a6a54-4fe3-4969-a9e3-404a37d4122d admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:23.028552 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a6a54-4fe3-4969-a9e3-404a37d4122d admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:23.028830 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a6a54-4fe3-4969-a9e3-404a37d4122d admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:23.029456 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011a6a54-4fe3-4969-a9e3-404a37d4122d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:23.031042 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011a6a54-4fe3-4969-a9e3-404a37d4122d admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:23.040903 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1355/2727] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:58:23.045922 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55162e-f801-4c2a-bab1-edf8825635a3 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:23.056132 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55162e-f801-4c2a-bab1-edf8825635a3 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:23.061219 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65ad6a1-d7ce-44cb-b143-072ec20862f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:23.061859 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ad6a1-d7ce-44cb-b143-072ec20862f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:23.062138 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ad6a1-d7ce-44cb-b143-072ec20862f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:23.062411 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ad6a1-d7ce-44cb-b143-072ec20862f0 admin admin] PATH_INFO: `/v3/projects/7fedb037aba0465c9f3dd7e441f17cb3/users/add52b3a7d5e4992b14116d258ebbaba/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:23.064414 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65ad6a1-d7ce-44cb-b143-072ec20862f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fedb037aba0465c9f3dd7e441f17cb3, user_id=add52b3a7d5e4992b14116d258ebbaba, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:23.067001 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65ad6a1-d7ce-44cb-b143-072ec20862f0 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:23.077214 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1373/2728] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:58:23 2022] PUT /identity/v3/projects/7fedb037aba0465c9f3dd7e441f17cb3/users/add52b3a7d5e4992b14116d258ebbaba/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:23.088166 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc1a4c-144e-4aa3-804e-73785c2064c8 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:23.088459 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc1a4c-144e-4aa3-804e-73785c2064c8 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:23.088738 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc1a4c-144e-4aa3-804e-73785c2064c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:23.118603 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-16fc1a4c-144e-4aa3-804e-73785c2064c8 None None] MFA Rules not processed for user `add52b3a7d5e4992b14116d258ebbaba`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:58:23.119771 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-16fc1a4c-144e-4aa3-804e-73785c2064c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:23.145794 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:23.187158 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1356/2729] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:58:23 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:58:23.197719 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65ad6a1-d7ce-44cb-b143-072ec20862f0 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:23.199441 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-f65ad6a1-d7ce-44cb-b143-072ec20862f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:23.263339 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65ad6a1-d7ce-44cb-b143-072ec20862f0 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:23.267067 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1473e7-68f5-4c44-8199-7541d63121d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:23.267718 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1473e7-68f5-4c44-8199-7541d63121d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:23.268001 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1473e7-68f5-4c44-8199-7541d63121d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:23.268272 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1473e7-68f5-4c44-8199-7541d63121d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:23.279981 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1473e7-68f5-4c44-8199-7541d63121d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:23.281407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1473e7-68f5-4c44-8199-7541d63121d7 service cinder] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:23.290914 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1374/2730] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:58:23 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:58:23.430684 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fc1a4c-144e-4aa3-804e-73785c2064c8 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:23.495943 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fc1a4c-144e-4aa3-804e-73785c2064c8 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:23.501470 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f560724d-3b94-4fe9-b08c-48bb8928c4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:23.502096 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560724d-3b94-4fe9-b08c-48bb8928c4db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:23.502397 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560724d-3b94-4fe9-b08c-48bb8928c4db admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:23.502669 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560724d-3b94-4fe9-b08c-48bb8928c4db admin admin] PATH_INFO: `/v3/users/20817b3b0b2e4038a8ef17723ea7fd27` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:23.503793 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f560724d-3b94-4fe9-b08c-48bb8928c4db admin admin] RBAC: Authorizing `identity:delete_user(user_id=20817b3b0b2e4038a8ef17723ea7fd27)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:23.505691 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f560724d-3b94-4fe9-b08c-48bb8928c4db admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:23.677264 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-f560724d-3b94-4fe9-b08c-48bb8928c4db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20817b3b0b2e4038a8ef17723ea7fd27'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:23.691545 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-f560724d-3b94-4fe9-b08c-48bb8928c4db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20817b3b0b2e4038a8ef17723ea7fd27'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:23.692264 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-f560724d-3b94-4fe9-b08c-48bb8928c4db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20817b3b0b2e4038a8ef17723ea7fd27'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:23.699200 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-f560724d-3b94-4fe9-b08c-48bb8928c4db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20817b3b0b2e4038a8ef17723ea7fd27'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:23.767729 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-f560724d-3b94-4fe9-b08c-48bb8928c4db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20817b3b0b2e4038a8ef17723ea7fd27'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:23.774052 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1357/2731] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:58:23 2022] DELETE /identity/v3/users/20817b3b0b2e4038a8ef17723ea7fd27 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:23.779193 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e41c4-b28a-4ce7-87c4-72b0debc5648 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:23.779505 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e41c4-b28a-4ce7-87c4-72b0debc5648 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:23.779774 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e41c4-b28a-4ce7-87c4-72b0debc5648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:23.798966 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-475e41c4-b28a-4ce7-87c4-72b0debc5648 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:58:23.800190 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-475e41c4-b28a-4ce7-87c4-72b0debc5648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:23.820909 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:23.861094 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1375/2732] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:58:23.872105 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f560724d-3b94-4fe9-b08c-48bb8928c4db admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:23.873619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-f560724d-3b94-4fe9-b08c-48bb8928c4db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:23.922508 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:23.934520 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f560724d-3b94-4fe9-b08c-48bb8928c4db admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:23.938288 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a60ad0-c6f8-4079-9eb7-914a9850d3ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:23.938900 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a60ad0-c6f8-4079-9eb7-914a9850d3ea service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:23.939171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a60ad0-c6f8-4079-9eb7-914a9850d3ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:23.939468 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a60ad0-c6f8-4079-9eb7-914a9850d3ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:23.948392 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a60ad0-c6f8-4079-9eb7-914a9850d3ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:23.950426 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a60ad0-c6f8-4079-9eb7-914a9850d3ea service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:23.959845 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1358/2733] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:58:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:58:24.380554 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e41c4-b28a-4ce7-87c4-72b0debc5648 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:24.382436 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-475e41c4-b28a-4ce7-87c4-72b0debc5648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:24.399188 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e41c4-b28a-4ce7-87c4-72b0debc5648 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:24.407886 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f726b-a34e-4083-88f9-60c4b193b7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:24.408562 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f726b-a34e-4083-88f9-60c4b193b7bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:24.408846 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f726b-a34e-4083-88f9-60c4b193b7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:24.409121 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f726b-a34e-4083-88f9-60c4b193b7bf admin admin] PATH_INFO: `/v3/projects/807a5f053ed742e38ca2c23d1e9e577f` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:24.410245 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f726b-a34e-4083-88f9-60c4b193b7bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=807a5f053ed742e38ca2c23d1e9e577f)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:24.412216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f726b-a34e-4083-88f9-60c4b193b7bf admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:24.495606 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-598f726b-a34e-4083-88f9-60c4b193b7bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '807a5f053ed742e38ca2c23d1e9e577f', 'request_id': 'req-598f726b-a34e-4083-88f9-60c4b193b7bf'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:24.501944 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-598f726b-a34e-4083-88f9-60c4b193b7bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '807a5f053ed742e38ca2c23d1e9e577f', 'request_id': 'req-598f726b-a34e-4083-88f9-60c4b193b7bf'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:24.608460 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-598f726b-a34e-4083-88f9-60c4b193b7bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '807a5f053ed742e38ca2c23d1e9e577f', 'request_id': 'req-598f726b-a34e-4083-88f9-60c4b193b7bf'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:24.620070 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-598f726b-a34e-4083-88f9-60c4b193b7bf admin admin] The token cache is being invalidate because project 807a5f053ed742e38ca2c23d1e9e577f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:58:24.620359 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-598f726b-a34e-4083-88f9-60c4b193b7bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:24.621001 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-598f726b-a34e-4083-88f9-60c4b193b7bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:24.622498 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1376/2734] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:58:24 2022] DELETE /identity/v3/projects/807a5f053ed742e38ca2c23d1e9e577f => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:24.626510 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a60ad0-c6f8-4079-9eb7-914a9850d3ea service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:24.637285 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a60ad0-c6f8-4079-9eb7-914a9850d3ea service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:24.695583 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682c491e-c492-4181-8d9c-79470a280e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:24.696271 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c491e-c492-4181-8d9c-79470a280e1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:24.696474 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c491e-c492-4181-8d9c-79470a280e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:24.696746 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c491e-c492-4181-8d9c-79470a280e1e admin admin] PATH_INFO: `/v3/users/add52b3a7d5e4992b14116d258ebbaba` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:24.697926 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682c491e-c492-4181-8d9c-79470a280e1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=add52b3a7d5e4992b14116d258ebbaba)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:24.700625 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682c491e-c492-4181-8d9c-79470a280e1e admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:24.888545 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-682c491e-c492-4181-8d9c-79470a280e1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'add52b3a7d5e4992b14116d258ebbaba'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:24.900828 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-682c491e-c492-4181-8d9c-79470a280e1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'add52b3a7d5e4992b14116d258ebbaba'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:24.901496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-682c491e-c492-4181-8d9c-79470a280e1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'add52b3a7d5e4992b14116d258ebbaba'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:24.909588 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-682c491e-c492-4181-8d9c-79470a280e1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'add52b3a7d5e4992b14116d258ebbaba'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:24.973517 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-682c491e-c492-4181-8d9c-79470a280e1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'add52b3a7d5e4992b14116d258ebbaba'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:24.980076 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1359/2735] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:58:24 2022] DELETE /identity/v3/users/add52b3a7d5e4992b14116d258ebbaba => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:25.359674 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f726b-a34e-4083-88f9-60c4b193b7bf admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:25.360426 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-598f726b-a34e-4083-88f9-60c4b193b7bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:25.373790 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f726b-a34e-4083-88f9-60c4b193b7bf admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:25.430804 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824467ef-95be-4a7a-86ee-f060acaa794c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:25.431394 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824467ef-95be-4a7a-86ee-f060acaa794c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:25.431679 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824467ef-95be-4a7a-86ee-f060acaa794c admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:25.431955 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824467ef-95be-4a7a-86ee-f060acaa794c admin admin] PATH_INFO: `/v3/projects/7fedb037aba0465c9f3dd7e441f17cb3` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:25.433057 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824467ef-95be-4a7a-86ee-f060acaa794c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fedb037aba0465c9f3dd7e441f17cb3)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:25.435192 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824467ef-95be-4a7a-86ee-f060acaa794c admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:25.533567 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-824467ef-95be-4a7a-86ee-f060acaa794c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fedb037aba0465c9f3dd7e441f17cb3', 'request_id': 'req-824467ef-95be-4a7a-86ee-f060acaa794c'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:25.535460 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:25.539980 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-824467ef-95be-4a7a-86ee-f060acaa794c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fedb037aba0465c9f3dd7e441f17cb3', 'request_id': 'req-824467ef-95be-4a7a-86ee-f060acaa794c'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:25.636305 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-824467ef-95be-4a7a-86ee-f060acaa794c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fedb037aba0465c9f3dd7e441f17cb3', 'request_id': 'req-824467ef-95be-4a7a-86ee-f060acaa794c'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:25.648450 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-824467ef-95be-4a7a-86ee-f060acaa794c admin admin] The token cache is being invalidate because project 7fedb037aba0465c9f3dd7e441f17cb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:58:25.648778 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-824467ef-95be-4a7a-86ee-f060acaa794c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:25.649403 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-824467ef-95be-4a7a-86ee-f060acaa794c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:25.650907 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1377/2736] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:58:25 2022] DELETE /identity/v3/projects/7fedb037aba0465c9f3dd7e441f17cb3 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:25.655701 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f70f37-882f-4cc7-a7f7-84a92ad4bc64 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:25.656005 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f70f37-882f-4cc7-a7f7-84a92ad4bc64 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:25.656263 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f70f37-882f-4cc7-a7f7-84a92ad4bc64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:25.677485 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-06f70f37-882f-4cc7-a7f7-84a92ad4bc64 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:58:25.678678 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-06f70f37-882f-4cc7-a7f7-84a92ad4bc64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:25.737433 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1360/2737] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:58:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:58:25.745238 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2842085-414e-4579-bab7-fe65decf03d8 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:25.745527 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2842085-414e-4579-bab7-fe65decf03d8 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:25.745801 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2842085-414e-4579-bab7-fe65decf03d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:25.765964 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-d2842085-414e-4579-bab7-fe65decf03d8 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:58:25.767158 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-d2842085-414e-4579-bab7-fe65decf03d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:25.775125 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1378/2738] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:58:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:58:25.779448 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f70f37-882f-4cc7-a7f7-84a92ad4bc64 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:25.788786 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:25.789004 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f70f37-882f-4cc7-a7f7-84a92ad4bc64 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:25.794129 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa1e2b1-5742-4010-8853-df7c5ef33a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:25.794727 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1e2b1-5742-4010-8853-df7c5ef33a92 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:25.795002 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1e2b1-5742-4010-8853-df7c5ef33a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:25.795268 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1e2b1-5742-4010-8853-df7c5ef33a92 admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:25.795936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa1e2b1-5742-4010-8853-df7c5ef33a92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:58:25.796234 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa1e2b1-5742-4010-8853-df7c5ef33a92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:25.797809 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa1e2b1-5742-4010-8853-df7c5ef33a92 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:25.809653 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1361/2739] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:58:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:58:25.813772 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2842085-414e-4579-bab7-fe65decf03d8 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:25.822908 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2842085-414e-4579-bab7-fe65decf03d8 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:25.827956 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c508760e-2c6d-4241-a687-e75cb550f431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:25.828552 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c508760e-2c6d-4241-a687-e75cb550f431 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:25.828831 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c508760e-2c6d-4241-a687-e75cb550f431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:25.829107 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c508760e-2c6d-4241-a687-e75cb550f431 admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:25.829736 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c508760e-2c6d-4241-a687-e75cb550f431 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:25.831657 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c508760e-2c6d-4241-a687-e75cb550f431 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:25.929015 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1379/2740] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:58:25 2022] POST /identity/v3/projects => generated 379 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:58:25.933327 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa1e2b1-5742-4010-8853-df7c5ef33a92 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:25.943465 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa1e2b1-5742-4010-8853-df7c5ef33a92 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:26.003901 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed4bd9-bf80-47fa-88ff-dfaaeade1938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:26.004505 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed4bd9-bf80-47fa-88ff-dfaaeade1938 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:26.004780 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed4bd9-bf80-47fa-88ff-dfaaeade1938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:26.005049 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed4bd9-bf80-47fa-88ff-dfaaeade1938 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:26.005673 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ed4bd9-bf80-47fa-88ff-dfaaeade1938 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:26.007629 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ed4bd9-bf80-47fa-88ff-dfaaeade1938 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:26.111178 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1362/2741] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:58:25 2022] POST /identity/v3/users => generated 290 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:58:26.115446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c508760e-2c6d-4241-a687-e75cb550f431 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:26.126598 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c508760e-2c6d-4241-a687-e75cb550f431 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:26.131235 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb299d-49f4-4704-b72d-3a5e42134a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:26.131869 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb299d-49f4-4704-b72d-3a5e42134a24 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:26.132150 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb299d-49f4-4704-b72d-3a5e42134a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:26.132426 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb299d-49f4-4704-b72d-3a5e42134a24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:26.133065 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb299d-49f4-4704-b72d-3a5e42134a24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:26.134652 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb299d-49f4-4704-b72d-3a5e42134a24 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:26.144883 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1380/2742] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:58:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:58:26.148701 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed4bd9-bf80-47fa-88ff-dfaaeade1938 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:26.158917 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed4bd9-bf80-47fa-88ff-dfaaeade1938 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:26.163480 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c238215-3903-40f4-b2db-dbf0bbfeeb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:26.164065 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c238215-3903-40f4-b2db-dbf0bbfeeb73 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:26.164342 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c238215-3903-40f4-b2db-dbf0bbfeeb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:26.164614 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c238215-3903-40f4-b2db-dbf0bbfeeb73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:26.165240 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c238215-3903-40f4-b2db-dbf0bbfeeb73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:26.166803 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c238215-3903-40f4-b2db-dbf0bbfeeb73 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:26.177959 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1363/2743] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:58:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:58:26.182011 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb299d-49f4-4704-b72d-3a5e42134a24 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:26.191993 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb299d-49f4-4704-b72d-3a5e42134a24 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:26.196505 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3e382d-8027-4565-b070-0aa1f20e4f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:26.197146 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e382d-8027-4565-b070-0aa1f20e4f60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:26.197421 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e382d-8027-4565-b070-0aa1f20e4f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:26.197690 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e382d-8027-4565-b070-0aa1f20e4f60 admin admin] PATH_INFO: `/v3/projects/003dee74d3014af0b9e94ac91cb4a6b5/users/1ee07af903d3481ea214dcf89f739b09/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:26.212196 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3e382d-8027-4565-b070-0aa1f20e4f60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=003dee74d3014af0b9e94ac91cb4a6b5, user_id=1ee07af903d3481ea214dcf89f739b09, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:26.214796 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3e382d-8027-4565-b070-0aa1f20e4f60 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:26.282895 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1381/2744] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:58:26 2022] PUT /identity/v3/projects/003dee74d3014af0b9e94ac91cb4a6b5/users/1ee07af903d3481ea214dcf89f739b09/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:26.293078 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dae80e-8ff4-4b62-9f00-071a4bfd543c None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:26.293517 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dae80e-8ff4-4b62-9f00-071a4bfd543c None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:26.293835 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dae80e-8ff4-4b62-9f00-071a4bfd543c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:26.320819 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-99dae80e-8ff4-4b62-9f00-071a4bfd543c None None] MFA Rules not processed for user `1ee07af903d3481ea214dcf89f739b09`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:58:26.322004 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-99dae80e-8ff4-4b62-9f00-071a4bfd543c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:26.386310 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1364/2745] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:58:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:58:26.391431 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3e382d-8027-4565-b070-0aa1f20e4f60 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:26.453127 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3e382d-8027-4565-b070-0aa1f20e4f60 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:26.457732 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eb205c-6f87-4510-89b1-382d1ffb73ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:26.458331 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb205c-6f87-4510-89b1-382d1ffb73ca admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:26.458617 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb205c-6f87-4510-89b1-382d1ffb73ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:26.458893 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb205c-6f87-4510-89b1-382d1ffb73ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:26.459571 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eb205c-6f87-4510-89b1-382d1ffb73ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:26.461457 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eb205c-6f87-4510-89b1-382d1ffb73ca admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:26.541220 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1382/2746] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:58:26 2022] POST /identity/v3/projects => generated 381 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:58:26.544552 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dae80e-8ff4-4b62-9f00-071a4bfd543c None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:26.555224 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dae80e-8ff4-4b62-9f00-071a4bfd543c None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:26.611653 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40df682-ef14-43b4-948c-4812eace50f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:26.612268 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40df682-ef14-43b4-948c-4812eace50f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:26.612546 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40df682-ef14-43b4-948c-4812eace50f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:26.612817 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40df682-ef14-43b4-948c-4812eace50f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:26.613451 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40df682-ef14-43b4-948c-4812eace50f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:26.615383 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40df682-ef14-43b4-948c-4812eace50f8 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:26.624214 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:26.814725 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1365/2747] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:58:26 2022] POST /identity/v3/users => generated 297 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:58:26.818717 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eb205c-6f87-4510-89b1-382d1ffb73ca admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:26.837099 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eb205c-6f87-4510-89b1-382d1ffb73ca admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:26.842054 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86617acf-590d-49a8-b4ac-b9d6e69baa0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:26.842661 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86617acf-590d-49a8-b4ac-b9d6e69baa0a admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:26.842947 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86617acf-590d-49a8-b4ac-b9d6e69baa0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:26.843225 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86617acf-590d-49a8-b4ac-b9d6e69baa0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:26.843925 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86617acf-590d-49a8-b4ac-b9d6e69baa0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:26.845521 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86617acf-590d-49a8-b4ac-b9d6e69baa0a admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:26.856876 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1383/2748] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:58:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 12:58:26.861017 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40df682-ef14-43b4-948c-4812eace50f8 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:26.872883 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40df682-ef14-43b4-948c-4812eace50f8 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:26.877707 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9773e09a-c12c-44ff-bd2a-2085fa1556d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:26.878362 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9773e09a-c12c-44ff-bd2a-2085fa1556d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:26.878676 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9773e09a-c12c-44ff-bd2a-2085fa1556d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:26.878952 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9773e09a-c12c-44ff-bd2a-2085fa1556d9 admin admin] PATH_INFO: `/v3/projects/a8871f96e46746bb89fd850ce4aa99e0/users/0920b888d22f40618278aeb6b778114f/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:26.894829 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9773e09a-c12c-44ff-bd2a-2085fa1556d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8871f96e46746bb89fd850ce4aa99e0, user_id=0920b888d22f40618278aeb6b778114f, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:26.897554 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9773e09a-c12c-44ff-bd2a-2085fa1556d9 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:26.960388 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1366/2749] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:58:26 2022] PUT /identity/v3/projects/a8871f96e46746bb89fd850ce4aa99e0/users/0920b888d22f40618278aeb6b778114f/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:26.964649 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86617acf-590d-49a8-b4ac-b9d6e69baa0a admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:26.976673 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86617acf-590d-49a8-b4ac-b9d6e69baa0a admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:27.030101 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:27.039833 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78113f53-5a6d-43c6-9ef7-59faba581096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:27.040512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78113f53-5a6d-43c6-9ef7-59faba581096 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:27.040798 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78113f53-5a6d-43c6-9ef7-59faba581096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:27.041075 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78113f53-5a6d-43c6-9ef7-59faba581096 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:27.041711 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78113f53-5a6d-43c6-9ef7-59faba581096 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:27.043826 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78113f53-5a6d-43c6-9ef7-59faba581096 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:27.053703 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1384/2750] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:58:26 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:58:27.058274 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9773e09a-c12c-44ff-bd2a-2085fa1556d9 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:27.069702 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9773e09a-c12c-44ff-bd2a-2085fa1556d9 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:27.074327 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a27f49-05a4-4274-8240-7609ebc8b248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:27.075136 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a27f49-05a4-4274-8240-7609ebc8b248 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:27.075514 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a27f49-05a4-4274-8240-7609ebc8b248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:27.075858 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a27f49-05a4-4274-8240-7609ebc8b248 admin admin] PATH_INFO: `/v3/projects/a8871f96e46746bb89fd850ce4aa99e0/users/0920b888d22f40618278aeb6b778114f/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:27.077905 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a27f49-05a4-4274-8240-7609ebc8b248 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8871f96e46746bb89fd850ce4aa99e0, user_id=0920b888d22f40618278aeb6b778114f, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:27.080929 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a27f49-05a4-4274-8240-7609ebc8b248 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:27.091910 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1367/2751] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:58:27 2022] PUT /identity/v3/projects/a8871f96e46746bb89fd850ce4aa99e0/users/0920b888d22f40618278aeb6b778114f/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:27.162269 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1435bb1-e128-4f2c-83fb-52ac406eacfe None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:27.162564 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1435bb1-e128-4f2c-83fb-52ac406eacfe None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:27.162848 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1435bb1-e128-4f2c-83fb-52ac406eacfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:27.192462 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-f1435bb1-e128-4f2c-83fb-52ac406eacfe None None] MFA Rules not processed for user `0920b888d22f40618278aeb6b778114f`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:58:27.193680 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-f1435bb1-e128-4f2c-83fb-52ac406eacfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:27.266358 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1385/2752] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:58:27 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:58:27.278419 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a27f49-05a4-4274-8240-7609ebc8b248 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:27.280467 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-28a27f49-05a4-4274-8240-7609ebc8b248 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:27.339573 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a27f49-05a4-4274-8240-7609ebc8b248 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:27.344324 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426d9af-9ed1-4d8c-8418-802cc3183774 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:27.344964 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426d9af-9ed1-4d8c-8418-802cc3183774 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:27.345263 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426d9af-9ed1-4d8c-8418-802cc3183774 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:27.345524 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426d9af-9ed1-4d8c-8418-802cc3183774 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:27.354953 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b426d9af-9ed1-4d8c-8418-802cc3183774 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:27.356613 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b426d9af-9ed1-4d8c-8418-802cc3183774 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:27.368240 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1368/2753] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:58:27 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:58:27.890558 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1435bb1-e128-4f2c-83fb-52ac406eacfe None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:27.953285 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1435bb1-e128-4f2c-83fb-52ac406eacfe None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:27.957865 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e9e86a-8dc9-4bea-8877-21ab785b4430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:27.958497 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e9e86a-8dc9-4bea-8877-21ab785b4430 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:27.958779 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e9e86a-8dc9-4bea-8877-21ab785b4430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:27.959056 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e9e86a-8dc9-4bea-8877-21ab785b4430 admin admin] PATH_INFO: `/v3/users/1ee07af903d3481ea214dcf89f739b09` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:27.960201 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e9e86a-8dc9-4bea-8877-21ab785b4430 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ee07af903d3481ea214dcf89f739b09)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:27.962113 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e9e86a-8dc9-4bea-8877-21ab785b4430 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:28.130741 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-e4e9e86a-8dc9-4bea-8877-21ab785b4430 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ee07af903d3481ea214dcf89f739b09'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:28.138431 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-e4e9e86a-8dc9-4bea-8877-21ab785b4430 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ee07af903d3481ea214dcf89f739b09'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:28.150333 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-e4e9e86a-8dc9-4bea-8877-21ab785b4430 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ee07af903d3481ea214dcf89f739b09'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:28.151004 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-e4e9e86a-8dc9-4bea-8877-21ab785b4430 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ee07af903d3481ea214dcf89f739b09'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:28.152746 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:28.224145 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-e4e9e86a-8dc9-4bea-8877-21ab785b4430 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ee07af903d3481ea214dcf89f739b09'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:28.230325 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1386/2754] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:58:27 2022] DELETE /identity/v3/users/1ee07af903d3481ea214dcf89f739b09 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:28.234832 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af8643e-6504-4130-a537-e8745a2bf82b None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:28.235334 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af8643e-6504-4130-a537-e8745a2bf82b None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:28.235469 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af8643e-6504-4130-a537-e8745a2bf82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:28.256033 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-3af8643e-6504-4130-a537-e8745a2bf82b None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:58:28.257238 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-3af8643e-6504-4130-a537-e8745a2bf82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:28.316428 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1369/2755] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:58:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:58:28.327251 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e9e86a-8dc9-4bea-8877-21ab785b4430 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:28.328750 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-e4e9e86a-8dc9-4bea-8877-21ab785b4430 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:28.404876 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e9e86a-8dc9-4bea-8877-21ab785b4430 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:28.408452 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87780ebc-83aa-4609-b0ab-d56f82c8268f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:28.409061 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87780ebc-83aa-4609-b0ab-d56f82c8268f service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:28.409346 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87780ebc-83aa-4609-b0ab-d56f82c8268f service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:28.409612 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87780ebc-83aa-4609-b0ab-d56f82c8268f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:28.419468 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87780ebc-83aa-4609-b0ab-d56f82c8268f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:28.422056 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87780ebc-83aa-4609-b0ab-d56f82c8268f service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:28.430503 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1387/2756] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:58:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:58:28.861429 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af8643e-6504-4130-a537-e8745a2bf82b None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:28.862980 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-3af8643e-6504-4130-a537-e8745a2bf82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:28.877707 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af8643e-6504-4130-a537-e8745a2bf82b None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:28.880594 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ba98a-761f-4d77-85d7-6f6d02fb379c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:28.881232 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ba98a-761f-4d77-85d7-6f6d02fb379c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:28.881511 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ba98a-761f-4d77-85d7-6f6d02fb379c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:28.881810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ba98a-761f-4d77-85d7-6f6d02fb379c admin admin] PATH_INFO: `/v3/projects/003dee74d3014af0b9e94ac91cb4a6b5` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:28.882959 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77ba98a-761f-4d77-85d7-6f6d02fb379c admin admin] RBAC: Authorizing `identity:delete_project(project_id=003dee74d3014af0b9e94ac91cb4a6b5)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:28.884929 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77ba98a-761f-4d77-85d7-6f6d02fb379c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:29.002349 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b77ba98a-761f-4d77-85d7-6f6d02fb379c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '003dee74d3014af0b9e94ac91cb4a6b5', 'request_id': 'req-b77ba98a-761f-4d77-85d7-6f6d02fb379c'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:29.003576 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:INSERT=5,keystone:SELECT=91,keystone:DELETE=4 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:29.008371 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b77ba98a-761f-4d77-85d7-6f6d02fb379c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '003dee74d3014af0b9e94ac91cb4a6b5', 'request_id': 'req-b77ba98a-761f-4d77-85d7-6f6d02fb379c'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:29.047438 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b77ba98a-761f-4d77-85d7-6f6d02fb379c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '003dee74d3014af0b9e94ac91cb4a6b5', 'request_id': 'req-b77ba98a-761f-4d77-85d7-6f6d02fb379c'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:29.059803 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b77ba98a-761f-4d77-85d7-6f6d02fb379c admin admin] The token cache is being invalidate because project 003dee74d3014af0b9e94ac91cb4a6b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:58:29.060090 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b77ba98a-761f-4d77-85d7-6f6d02fb379c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:29.060726 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-b77ba98a-761f-4d77-85d7-6f6d02fb379c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:29.062201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1370/2757] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:58:28 2022] DELETE /identity/v3/projects/003dee74d3014af0b9e94ac91cb4a6b5 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:29.066449 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87780ebc-83aa-4609-b0ab-d56f82c8268f service neutron] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:29.076256 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87780ebc-83aa-4609-b0ab-d56f82c8268f service neutron] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:29.137612 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8927a91-8bdd-4781-9e6a-d0957675e699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:29.138285 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8927a91-8bdd-4781-9e6a-d0957675e699 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:29.138566 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8927a91-8bdd-4781-9e6a-d0957675e699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:29.138842 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8927a91-8bdd-4781-9e6a-d0957675e699 admin admin] PATH_INFO: `/v3/users/0920b888d22f40618278aeb6b778114f` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:29.140003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8927a91-8bdd-4781-9e6a-d0957675e699 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0920b888d22f40618278aeb6b778114f)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:29.141916 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8927a91-8bdd-4781-9e6a-d0957675e699 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:29.308313 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-e8927a91-8bdd-4781-9e6a-d0957675e699 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0920b888d22f40618278aeb6b778114f'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:29.315730 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-e8927a91-8bdd-4781-9e6a-d0957675e699 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0920b888d22f40618278aeb6b778114f'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:29.328449 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-e8927a91-8bdd-4781-9e6a-d0957675e699 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0920b888d22f40618278aeb6b778114f'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:29.329178 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-e8927a91-8bdd-4781-9e6a-d0957675e699 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0920b888d22f40618278aeb6b778114f'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:29.430543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-e8927a91-8bdd-4781-9e6a-d0957675e699 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0920b888d22f40618278aeb6b778114f'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:29.437867 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1388/2758] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:58:29 2022] DELETE /identity/v3/users/0920b888d22f40618278aeb6b778114f => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:29.843556 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ba98a-761f-4d77-85d7-6f6d02fb379c admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:29.845140 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-b77ba98a-761f-4d77-85d7-6f6d02fb379c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:29.859668 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ba98a-761f-4d77-85d7-6f6d02fb379c admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:29.943590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd99d329-4646-4451-96e9-b81aa1ab0992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:29.944162 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99d329-4646-4451-96e9-b81aa1ab0992 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:29.944466 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99d329-4646-4451-96e9-b81aa1ab0992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:29.944701 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99d329-4646-4451-96e9-b81aa1ab0992 admin admin] PATH_INFO: `/v3/projects/a8871f96e46746bb89fd850ce4aa99e0` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:29.945798 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd99d329-4646-4451-96e9-b81aa1ab0992 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8871f96e46746bb89fd850ce4aa99e0)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:29.947732 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd99d329-4646-4451-96e9-b81aa1ab0992 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:30.080264 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-dd99d329-4646-4451-96e9-b81aa1ab0992 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8871f96e46746bb89fd850ce4aa99e0', 'request_id': 'req-dd99d329-4646-4451-96e9-b81aa1ab0992'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:30.087236 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-dd99d329-4646-4451-96e9-b81aa1ab0992 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8871f96e46746bb89fd850ce4aa99e0', 'request_id': 'req-dd99d329-4646-4451-96e9-b81aa1ab0992'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:30.190351 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-dd99d329-4646-4451-96e9-b81aa1ab0992 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8871f96e46746bb89fd850ce4aa99e0', 'request_id': 'req-dd99d329-4646-4451-96e9-b81aa1ab0992'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:30.202731 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-dd99d329-4646-4451-96e9-b81aa1ab0992 admin admin] The token cache is being invalidate because project a8871f96e46746bb89fd850ce4aa99e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:58:30.202998 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-dd99d329-4646-4451-96e9-b81aa1ab0992 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:30.203659 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-dd99d329-4646-4451-96e9-b81aa1ab0992 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:30.205189 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1371/2759] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:58:29 2022] DELETE /identity/v3/projects/a8871f96e46746bb89fd850ce4aa99e0 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:30.210135 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b7528-1855-4676-b94b-9cc2910402bc None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:30.210639 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b7528-1855-4676-b94b-9cc2910402bc None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:30.210741 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b7528-1855-4676-b94b-9cc2910402bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:30.230140 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-ef3b7528-1855-4676-b94b-9cc2910402bc None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:58:30.232114 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-ef3b7528-1855-4676-b94b-9cc2910402bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:30.241420 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:30.298463 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1389/2760] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:58:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:58:30.306634 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03043a1-1a67-4c14-8656-e877288ac5f7 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:30.306853 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03043a1-1a67-4c14-8656-e877288ac5f7 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:30.307126 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03043a1-1a67-4c14-8656-e877288ac5f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:30.328757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-a03043a1-1a67-4c14-8656-e877288ac5f7 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:58:30.329989 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a03043a1-1a67-4c14-8656-e877288ac5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:30.338389 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1372/2761] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:58:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:58:30.343469 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b7528-1855-4676-b94b-9cc2910402bc None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:30.353473 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b7528-1855-4676-b94b-9cc2910402bc None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:30.359129 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0632020-6c86-4d2e-938c-2f402aee4876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:30.359825 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0632020-6c86-4d2e-938c-2f402aee4876 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:30.360073 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0632020-6c86-4d2e-938c-2f402aee4876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:30.360349 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0632020-6c86-4d2e-938c-2f402aee4876 admin admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:30.361000 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0632020-6c86-4d2e-938c-2f402aee4876 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:58:30.361307 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0632020-6c86-4d2e-938c-2f402aee4876 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:30.362981 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0632020-6c86-4d2e-938c-2f402aee4876 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:30.377629 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1390/2762] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:58:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:58:30.383735 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03043a1-1a67-4c14-8656-e877288ac5f7 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:30.393849 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03043a1-1a67-4c14-8656-e877288ac5f7 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:30.399182 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d6359-a265-4cfb-94c5-613c44d69e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:30.399957 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d6359-a265-4cfb-94c5-613c44d69e1f admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:30.400234 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d6359-a265-4cfb-94c5-613c44d69e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:30.400512 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d6359-a265-4cfb-94c5-613c44d69e1f admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:30.401289 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5d6359-a265-4cfb-94c5-613c44d69e1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:30.403946 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5d6359-a265-4cfb-94c5-613c44d69e1f admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:30.479209 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1373/2763] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:58:30 2022] POST /identity/v3/projects => generated 371 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:58:30.483970 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0632020-6c86-4d2e-938c-2f402aee4876 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:30.494355 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0632020-6c86-4d2e-938c-2f402aee4876 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:30.556570 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4cc93b-6ffe-49c3-9cd9-733416ce7296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:30.557434 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4cc93b-6ffe-49c3-9cd9-733416ce7296 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:30.557554 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4cc93b-6ffe-49c3-9cd9-733416ce7296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:30.557849 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4cc93b-6ffe-49c3-9cd9-733416ce7296 admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:30.558496 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4cc93b-6ffe-49c3-9cd9-733416ce7296 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:30.562097 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4cc93b-6ffe-49c3-9cd9-733416ce7296 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:30.657298 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1391/2764] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:58:30 2022] POST /identity/v3/users => generated 286 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:58:30.658515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d6359-a265-4cfb-94c5-613c44d69e1f admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:30.670117 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d6359-a265-4cfb-94c5-613c44d69e1f admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:30.675161 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3beffb-2600-4bd3-86be-75fc8ec31eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:30.675812 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3beffb-2600-4bd3-86be-75fc8ec31eea admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:30.682114 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3beffb-2600-4bd3-86be-75fc8ec31eea admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:30.682114 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3beffb-2600-4bd3-86be-75fc8ec31eea admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:30.682114 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3beffb-2600-4bd3-86be-75fc8ec31eea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:30.682114 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3beffb-2600-4bd3-86be-75fc8ec31eea admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:30.692539 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1374/2765] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:58:30.693750 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4cc93b-6ffe-49c3-9cd9-733416ce7296 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:30.710938 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4cc93b-6ffe-49c3-9cd9-733416ce7296 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:30.711741 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec18325c-f035-44c6-af29-101fb7608492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:30.712353 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec18325c-f035-44c6-af29-101fb7608492 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:30.712632 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec18325c-f035-44c6-af29-101fb7608492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:30.712902 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec18325c-f035-44c6-af29-101fb7608492 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:30.713540 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec18325c-f035-44c6-af29-101fb7608492 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:30.716465 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec18325c-f035-44c6-af29-101fb7608492 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:30.727762 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1392/2766] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:58:30.734034 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3beffb-2600-4bd3-86be-75fc8ec31eea admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:30.743132 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3beffb-2600-4bd3-86be-75fc8ec31eea admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:30.748515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e65a77-c5d7-4bb5-a17c-f94bc53ad340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:30.749243 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e65a77-c5d7-4bb5-a17c-f94bc53ad340 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:30.749551 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e65a77-c5d7-4bb5-a17c-f94bc53ad340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:30.749830 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e65a77-c5d7-4bb5-a17c-f94bc53ad340 admin admin] PATH_INFO: `/v3/projects/bb707955dbbf426094d6584c4f5a4a36/users/3fb0f756066a4b93822be43c019ebacd/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:30.766774 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e65a77-c5d7-4bb5-a17c-f94bc53ad340 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb707955dbbf426094d6584c4f5a4a36, user_id=3fb0f756066a4b93822be43c019ebacd, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:30.769555 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e65a77-c5d7-4bb5-a17c-f94bc53ad340 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:30.825145 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1375/2767] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:58:30 2022] PUT /identity/v3/projects/bb707955dbbf426094d6584c4f5a4a36/users/3fb0f756066a4b93822be43c019ebacd/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:30.830327 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e742a-5356-49d2-8dbe-b637ec9b78fb None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:30.830617 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e742a-5356-49d2-8dbe-b637ec9b78fb None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:30.830888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e742a-5356-49d2-8dbe-b637ec9b78fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:30.851664 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-720e742a-5356-49d2-8dbe-b637ec9b78fb None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:58:30.852825 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-720e742a-5356-49d2-8dbe-b637ec9b78fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:30.917992 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1393/2768] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:58:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:58:30.929362 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e65a77-c5d7-4bb5-a17c-f94bc53ad340 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:30.931590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d9e65a77-c5d7-4bb5-a17c-f94bc53ad340 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:30.939576 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:31.013877 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e65a77-c5d7-4bb5-a17c-f94bc53ad340 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:31.018170 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6863d2a-8f39-44a8-a032-e8d1ae116ad4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:31.018961 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6863d2a-8f39-44a8-a032-e8d1ae116ad4 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:31.019240 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6863d2a-8f39-44a8-a032-e8d1ae116ad4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:31.019561 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6863d2a-8f39-44a8-a032-e8d1ae116ad4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:31.029482 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6863d2a-8f39-44a8-a032-e8d1ae116ad4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:31.030904 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6863d2a-8f39-44a8-a032-e8d1ae116ad4 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:31.040928 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1376/2769] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:58:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:58:32.253808 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd0711-9b9e-48af-8214-9fb806626de1 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:32.254368 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd0711-9b9e-48af-8214-9fb806626de1 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:32.254368 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd0711-9b9e-48af-8214-9fb806626de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:32.288269 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-22bd0711-9b9e-48af-8214-9fb806626de1 None None] MFA Rules not processed for user `3fb0f756066a4b93822be43c019ebacd`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:58:32.289868 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-22bd0711-9b9e-48af-8214-9fb806626de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:32.299939 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:32.368101 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1394/2770] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:58:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:58:32.374138 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6863d2a-8f39-44a8-a032-e8d1ae116ad4 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:32.384357 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6863d2a-8f39-44a8-a032-e8d1ae116ad4 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:32.389572 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b31762f-9940-4006-9832-df7ed33acdce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:32.390154 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b31762f-9940-4006-9832-df7ed33acdce admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:32.390429 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b31762f-9940-4006-9832-df7ed33acdce admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:32.390697 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b31762f-9940-4006-9832-df7ed33acdce admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:32.391350 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b31762f-9940-4006-9832-df7ed33acdce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:32.393238 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b31762f-9940-4006-9832-df7ed33acdce admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:32.469753 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1377/2771] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:58:32 2022] POST /identity/v3/projects => generated 373 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:58:32.470913 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bd0711-9b9e-48af-8214-9fb806626de1 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:32.482137 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bd0711-9b9e-48af-8214-9fb806626de1 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:32.540593 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d54ffe-9689-470c-8190-fcf198ae1f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:32.541115 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d54ffe-9689-470c-8190-fcf198ae1f4b admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:32.541421 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d54ffe-9689-470c-8190-fcf198ae1f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:32.541717 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d54ffe-9689-470c-8190-fcf198ae1f4b admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:32.542339 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d54ffe-9689-470c-8190-fcf198ae1f4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:32.544509 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d54ffe-9689-470c-8190-fcf198ae1f4b admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:32.708252 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1395/2772] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:58:32 2022] POST /identity/v3/users => generated 293 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:58:32.709730 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b31762f-9940-4006-9832-df7ed33acdce admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:32.720370 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b31762f-9940-4006-9832-df7ed33acdce admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:32.725326 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd07e5e-b8f4-4312-832b-5362d234d74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:32.726122 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd07e5e-b8f4-4312-832b-5362d234d74f admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:32.726395 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd07e5e-b8f4-4312-832b-5362d234d74f admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:32.726667 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd07e5e-b8f4-4312-832b-5362d234d74f admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:32.727303 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd07e5e-b8f4-4312-832b-5362d234d74f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:32.728913 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd07e5e-b8f4-4312-832b-5362d234d74f admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:32.741810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1378/2773] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:58:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:58:32.747522 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d54ffe-9689-470c-8190-fcf198ae1f4b admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:32.757665 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d54ffe-9689-470c-8190-fcf198ae1f4b admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:32.763151 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5816df-e76b-4dc7-8c57-2a9462d03a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:32.763778 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5816df-e76b-4dc7-8c57-2a9462d03a6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:32.764136 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5816df-e76b-4dc7-8c57-2a9462d03a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:32.764485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5816df-e76b-4dc7-8c57-2a9462d03a6d admin admin] PATH_INFO: `/v3/projects/061fb57754614b55b714086d70128534/users/7649c0cebeed4b1d8cb9521ff501c037/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:32.780098 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5816df-e76b-4dc7-8c57-2a9462d03a6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=061fb57754614b55b714086d70128534, user_id=7649c0cebeed4b1d8cb9521ff501c037, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:32.782772 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5816df-e76b-4dc7-8c57-2a9462d03a6d admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:32.853150 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1396/2774] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:58:32 2022] PUT /identity/v3/projects/061fb57754614b55b714086d70128534/users/7649c0cebeed4b1d8cb9521ff501c037/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:32.857348 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd07e5e-b8f4-4312-832b-5362d234d74f admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:32.868019 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd07e5e-b8f4-4312-832b-5362d234d74f admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:32.930031 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabac150-823a-4c83-990e-b371c48ba636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:32.930650 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabac150-823a-4c83-990e-b371c48ba636 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:32.930935 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabac150-823a-4c83-990e-b371c48ba636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:32.931214 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabac150-823a-4c83-990e-b371c48ba636 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:32.931936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabac150-823a-4c83-990e-b371c48ba636 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:32.933566 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabac150-823a-4c83-990e-b371c48ba636 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:32.947507 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1379/2775] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:58:32 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 12:58:32.951617 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5816df-e76b-4dc7-8c57-2a9462d03a6d admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:32.963485 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5816df-e76b-4dc7-8c57-2a9462d03a6d admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:32.968337 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7875c52-cd61-4655-92ad-e0e5108c23f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:32.969003 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7875c52-cd61-4655-92ad-e0e5108c23f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:32.969286 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7875c52-cd61-4655-92ad-e0e5108c23f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:32.969568 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7875c52-cd61-4655-92ad-e0e5108c23f9 admin admin] PATH_INFO: `/v3/projects/061fb57754614b55b714086d70128534/users/7649c0cebeed4b1d8cb9521ff501c037/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:32.972006 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7875c52-cd61-4655-92ad-e0e5108c23f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=061fb57754614b55b714086d70128534, user_id=7649c0cebeed4b1d8cb9521ff501c037, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:32.974614 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7875c52-cd61-4655-92ad-e0e5108c23f9 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:32.993060 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1397/2776] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:58:32 2022] PUT /identity/v3/projects/061fb57754614b55b714086d70128534/users/7649c0cebeed4b1d8cb9521ff501c037/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:34.230843 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad283de-2b61-4be9-bc68-583e30b7ba77 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:34.231401 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad283de-2b61-4be9-bc68-583e30b7ba77 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:34.232516 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad283de-2b61-4be9-bc68-583e30b7ba77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:34.269543 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-bad283de-2b61-4be9-bc68-583e30b7ba77 None None] MFA Rules not processed for user `7649c0cebeed4b1d8cb9521ff501c037`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:58:34.270278 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-bad283de-2b61-4be9-bc68-583e30b7ba77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:34.336188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:34.345012 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1380/2777] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:58:34 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:58:34.356482 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7875c52-cd61-4655-92ad-e0e5108c23f9 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:34.358171 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-f7875c52-cd61-4655-92ad-e0e5108c23f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:34.420269 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7875c52-cd61-4655-92ad-e0e5108c23f9 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:34.424496 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0d1546-a7a9-478b-809e-de0ff912999f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:34.425166 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0d1546-a7a9-478b-809e-de0ff912999f service cinder] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:34.425447 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0d1546-a7a9-478b-809e-de0ff912999f service cinder] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:34.425720 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0d1546-a7a9-478b-809e-de0ff912999f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:34.440593 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:34.488524 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0d1546-a7a9-478b-809e-de0ff912999f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:34.489902 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0d1546-a7a9-478b-809e-de0ff912999f service cinder] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:34.500794 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1398/2778] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:58:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:58:36.229924 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad283de-2b61-4be9-bc68-583e30b7ba77 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:36.239637 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad283de-2b61-4be9-bc68-583e30b7ba77 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:36.243374 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6acf1bb-04de-4bbc-84f2-7cccb188bf44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:36.243985 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6acf1bb-04de-4bbc-84f2-7cccb188bf44 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:36.244260 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6acf1bb-04de-4bbc-84f2-7cccb188bf44 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:36.244546 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6acf1bb-04de-4bbc-84f2-7cccb188bf44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:36.253656 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6acf1bb-04de-4bbc-84f2-7cccb188bf44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:36.255071 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6acf1bb-04de-4bbc-84f2-7cccb188bf44 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:36.265037 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1381/2779] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:58:36 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:58:43.909556 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8282aea-8538-45e1-8ad4-f4a81c868556 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:43.910082 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8282aea-8538-45e1-8ad4-f4a81c868556 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:43.910184 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8282aea-8538-45e1-8ad4-f4a81c868556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:43.930407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-e8282aea-8538-45e1-8ad4-f4a81c868556 None None] MFA Rules not processed for user `5e29a4c0276247c286cff3eed5a1253a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:58:43.932776 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-e8282aea-8538-45e1-8ad4-f4a81c868556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:43.989356 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1399/2780] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:58:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:58:46.260203 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=8 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:51.848424 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6acf1bb-04de-4bbc-84f2-7cccb188bf44 service cinder] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:51.850812 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a6acf1bb-04de-4bbc-84f2-7cccb188bf44 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:51.925603 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6acf1bb-04de-4bbc-84f2-7cccb188bf44 service cinder] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:51.930715 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204ff3a4-69b9-4e3d-b835-6b0c6ea349d9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25eef64e92e9422ea62508c3eb0a197e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:51.931397 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ff3a4-69b9-4e3d-b835-6b0c6ea349d9 service placement] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:51.931627 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ff3a4-69b9-4e3d-b835-6b0c6ea349d9 service placement] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:51.931895 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ff3a4-69b9-4e3d-b835-6b0c6ea349d9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:51.934543 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-204ff3a4-69b9-4e3d-b835-6b0c6ea349d9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:51.946653 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204ff3a4-69b9-4e3d-b835-6b0c6ea349d9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:51.948263 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204ff3a4-69b9-4e3d-b835-6b0c6ea349d9 service placement] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:51.959218 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1382/2781] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:58:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:58:52.676651 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8282aea-8538-45e1-8ad4-f4a81c868556 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:52.743376 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8282aea-8538-45e1-8ad4-f4a81c868556 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:52.748234 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc3d7e5-799e-4296-bf69-65ff2a257695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:52.748865 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc3d7e5-799e-4296-bf69-65ff2a257695 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:52.749147 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc3d7e5-799e-4296-bf69-65ff2a257695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:52.749422 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc3d7e5-799e-4296-bf69-65ff2a257695 admin admin] PATH_INFO: `/v3/users/3fb0f756066a4b93822be43c019ebacd` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:52.750527 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc3d7e5-799e-4296-bf69-65ff2a257695 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fb0f756066a4b93822be43c019ebacd)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:52.752513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc3d7e5-799e-4296-bf69-65ff2a257695 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:52.935706 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-3cc3d7e5-799e-4296-bf69-65ff2a257695 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fb0f756066a4b93822be43c019ebacd'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:52.943727 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-3cc3d7e5-799e-4296-bf69-65ff2a257695 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fb0f756066a4b93822be43c019ebacd'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:52.949212 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:52.957690 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-3cc3d7e5-799e-4296-bf69-65ff2a257695 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fb0f756066a4b93822be43c019ebacd'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:52.958450 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-3cc3d7e5-799e-4296-bf69-65ff2a257695 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fb0f756066a4b93822be43c019ebacd'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:53.013097 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-3cc3d7e5-799e-4296-bf69-65ff2a257695 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fb0f756066a4b93822be43c019ebacd'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:53.019236 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1400/2782] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:58:52 2022] DELETE /identity/v3/users/3fb0f756066a4b93822be43c019ebacd => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:53.249742 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204ff3a4-69b9-4e3d-b835-6b0c6ea349d9 service placement] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:53.260335 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204ff3a4-69b9-4e3d-b835-6b0c6ea349d9 service placement] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:53.321188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a014e30d-e9b5-4635-bf68-4d8312ce72cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:53.321818 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014e30d-e9b5-4635-bf68-4d8312ce72cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:53.322099 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014e30d-e9b5-4635-bf68-4d8312ce72cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:53.322387 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014e30d-e9b5-4635-bf68-4d8312ce72cf admin admin] PATH_INFO: `/v3/projects/bb707955dbbf426094d6584c4f5a4a36` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:53.323519 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a014e30d-e9b5-4635-bf68-4d8312ce72cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb707955dbbf426094d6584c4f5a4a36)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:53.326120 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a014e30d-e9b5-4635-bf68-4d8312ce72cf admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:53.424251 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a014e30d-e9b5-4635-bf68-4d8312ce72cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb707955dbbf426094d6584c4f5a4a36', 'request_id': 'req-a014e30d-e9b5-4635-bf68-4d8312ce72cf'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:53.429656 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a014e30d-e9b5-4635-bf68-4d8312ce72cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb707955dbbf426094d6584c4f5a4a36', 'request_id': 'req-a014e30d-e9b5-4635-bf68-4d8312ce72cf'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:53.475104 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a014e30d-e9b5-4635-bf68-4d8312ce72cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb707955dbbf426094d6584c4f5a4a36', 'request_id': 'req-a014e30d-e9b5-4635-bf68-4d8312ce72cf'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:53.486853 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a014e30d-e9b5-4635-bf68-4d8312ce72cf admin admin] The token cache is being invalidate because project bb707955dbbf426094d6584c4f5a4a36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:58:53.487998 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a014e30d-e9b5-4635-bf68-4d8312ce72cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:53.487998 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a014e30d-e9b5-4635-bf68-4d8312ce72cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:53.489258 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1383/2783] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:58:53 2022] DELETE /identity/v3/projects/bb707955dbbf426094d6584c4f5a4a36 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:53.493711 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc3d7e5-799e-4296-bf69-65ff2a257695 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:53.495163 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-3cc3d7e5-799e-4296-bf69-65ff2a257695 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:53.508602 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc3d7e5-799e-4296-bf69-65ff2a257695 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:53.566254 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d641f52-2591-4a59-a490-26b01c718aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:53.566813 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d641f52-2591-4a59-a490-26b01c718aea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:53.567093 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d641f52-2591-4a59-a490-26b01c718aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:53.567392 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d641f52-2591-4a59-a490-26b01c718aea admin admin] PATH_INFO: `/v3/users/7649c0cebeed4b1d8cb9521ff501c037` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:53.568573 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d641f52-2591-4a59-a490-26b01c718aea admin admin] RBAC: Authorizing `identity:delete_user(user_id=7649c0cebeed4b1d8cb9521ff501c037)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:53.570480 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d641f52-2591-4a59-a490-26b01c718aea admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:53.816057 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-9d641f52-2591-4a59-a490-26b01c718aea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7649c0cebeed4b1d8cb9521ff501c037'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:53.822978 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-9d641f52-2591-4a59-a490-26b01c718aea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7649c0cebeed4b1d8cb9521ff501c037'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:53.835232 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-9d641f52-2591-4a59-a490-26b01c718aea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7649c0cebeed4b1d8cb9521ff501c037'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:53.836019 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-9d641f52-2591-4a59-a490-26b01c718aea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7649c0cebeed4b1d8cb9521ff501c037'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:53.915961 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-9d641f52-2591-4a59-a490-26b01c718aea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7649c0cebeed4b1d8cb9521ff501c037'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:53.923275 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1401/2784] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:58:53 2022] DELETE /identity/v3/users/7649c0cebeed4b1d8cb9521ff501c037 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:54.271733 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a014e30d-e9b5-4635-bf68-4d8312ce72cf admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:54.272738 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a014e30d-e9b5-4635-bf68-4d8312ce72cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:54.286875 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a014e30d-e9b5-4635-bf68-4d8312ce72cf admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:54.309899 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:54.350974 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838147a5-78ef-4d48-a175-377dee634e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:54.351873 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838147a5-78ef-4d48-a175-377dee634e7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:54.351873 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838147a5-78ef-4d48-a175-377dee634e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:54.352137 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838147a5-78ef-4d48-a175-377dee634e7c admin admin] PATH_INFO: `/v3/projects/061fb57754614b55b714086d70128534` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:54.353393 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838147a5-78ef-4d48-a175-377dee634e7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=061fb57754614b55b714086d70128534)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:54.355349 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838147a5-78ef-4d48-a175-377dee634e7c admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:54.454539 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-838147a5-78ef-4d48-a175-377dee634e7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '061fb57754614b55b714086d70128534', 'request_id': 'req-838147a5-78ef-4d48-a175-377dee634e7c'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:54.460191 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-838147a5-78ef-4d48-a175-377dee634e7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '061fb57754614b55b714086d70128534', 'request_id': 'req-838147a5-78ef-4d48-a175-377dee634e7c'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:54.498578 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-838147a5-78ef-4d48-a175-377dee634e7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '061fb57754614b55b714086d70128534', 'request_id': 'req-838147a5-78ef-4d48-a175-377dee634e7c'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:54.509850 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-838147a5-78ef-4d48-a175-377dee634e7c admin admin] The token cache is being invalidate because project 061fb57754614b55b714086d70128534 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:58:54.510596 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-838147a5-78ef-4d48-a175-377dee634e7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:54.510745 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-838147a5-78ef-4d48-a175-377dee634e7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:54.512246 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1384/2785] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:58:54 2022] DELETE /identity/v3/projects/061fb57754614b55b714086d70128534 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:58:54.517225 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e4029-65ee-41de-85ab-f7d91a712192 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:54.517514 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e4029-65ee-41de-85ab-f7d91a712192 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:54.517786 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e4029-65ee-41de-85ab-f7d91a712192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:54.537927 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-437e4029-65ee-41de-85ab-f7d91a712192 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:58:54.539120 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-437e4029-65ee-41de-85ab-f7d91a712192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:54.590029 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:54.598277 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1402/2786] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:58:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:58:54.607011 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86598039-ff7a-434b-b8d2-efea753161b6 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:54.607229 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86598039-ff7a-434b-b8d2-efea753161b6 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:54.607541 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86598039-ff7a-434b-b8d2-efea753161b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:54.628673 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-86598039-ff7a-434b-b8d2-efea753161b6 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:58:54.629859 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-86598039-ff7a-434b-b8d2-efea753161b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:54.638094 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1385/2787] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:58:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:58:54.642248 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e4029-65ee-41de-85ab-f7d91a712192 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:54.652686 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e4029-65ee-41de-85ab-f7d91a712192 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:54.657522 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e1751c-714c-49a3-9ccb-26426a065092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:54.658126 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e1751c-714c-49a3-9ccb-26426a065092 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:54.658407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e1751c-714c-49a3-9ccb-26426a065092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:54.658679 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e1751c-714c-49a3-9ccb-26426a065092 admin admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:54.659339 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e1751c-714c-49a3-9ccb-26426a065092 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:58:54.659659 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e1751c-714c-49a3-9ccb-26426a065092 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:54.661324 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e1751c-714c-49a3-9ccb-26426a065092 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:54.673472 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1403/2788] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:58:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:58:54.677718 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86598039-ff7a-434b-b8d2-efea753161b6 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:54.687809 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86598039-ff7a-434b-b8d2-efea753161b6 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:54.692477 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20106b65-827d-4851-9115-2a5bda641dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:54.694326 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20106b65-827d-4851-9115-2a5bda641dc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:54.694326 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20106b65-827d-4851-9115-2a5bda641dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:54.694326 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20106b65-827d-4851-9115-2a5bda641dc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:54.694326 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20106b65-827d-4851-9115-2a5bda641dc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:54.696215 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20106b65-827d-4851-9115-2a5bda641dc6 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:54.808608 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1386/2789] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:58:54 2022] POST /identity/v3/projects => generated 373 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:58:54.812758 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e1751c-714c-49a3-9ccb-26426a065092 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:54.822677 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e1751c-714c-49a3-9ccb-26426a065092 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:54.881038 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb7a36d-76eb-40a5-8189-03aa7107ebac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:54.881651 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb7a36d-76eb-40a5-8189-03aa7107ebac admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:54.881933 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb7a36d-76eb-40a5-8189-03aa7107ebac admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:54.882207 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb7a36d-76eb-40a5-8189-03aa7107ebac admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:54.882835 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb7a36d-76eb-40a5-8189-03aa7107ebac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:54.884779 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb7a36d-76eb-40a5-8189-03aa7107ebac admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:54.989259 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1404/2790] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:58:54 2022] POST /identity/v3/users => generated 287 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:58:54.989748 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20106b65-827d-4851-9115-2a5bda641dc6 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:55.000837 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20106b65-827d-4851-9115-2a5bda641dc6 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:55.007809 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195e28f4-d92e-41bf-8422-ae603c9b9ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:55.007809 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e28f4-d92e-41bf-8422-ae603c9b9ecf admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:55.007809 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e28f4-d92e-41bf-8422-ae603c9b9ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:55.007809 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e28f4-d92e-41bf-8422-ae603c9b9ecf admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:55.008201 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195e28f4-d92e-41bf-8422-ae603c9b9ecf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:55.008740 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195e28f4-d92e-41bf-8422-ae603c9b9ecf admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:55.018560 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1387/2791] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:58:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:58:55.022515 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb7a36d-76eb-40a5-8189-03aa7107ebac admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:55.032925 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb7a36d-76eb-40a5-8189-03aa7107ebac admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:55.037527 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb7ade-1f8d-4a8d-a973-27e20086978a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:55.038116 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb7ade-1f8d-4a8d-a973-27e20086978a admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:55.038397 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb7ade-1f8d-4a8d-a973-27e20086978a admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:55.038671 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb7ade-1f8d-4a8d-a973-27e20086978a admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:55.039363 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affb7ade-1f8d-4a8d-a973-27e20086978a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:55.040946 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affb7ade-1f8d-4a8d-a973-27e20086978a admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:55.051935 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1405/2792] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:58:55.055815 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195e28f4-d92e-41bf-8422-ae603c9b9ecf admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:55.065703 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195e28f4-d92e-41bf-8422-ae603c9b9ecf admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:55.070480 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30523e26-d9ea-47ac-b73a-31a2fe46ebac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:55.071158 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30523e26-d9ea-47ac-b73a-31a2fe46ebac admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:55.071475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30523e26-d9ea-47ac-b73a-31a2fe46ebac admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:55.071753 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30523e26-d9ea-47ac-b73a-31a2fe46ebac admin admin] PATH_INFO: `/v3/projects/ac78a09616cf46e5a4cd6297ed41ad88/users/87ecdefbce014a9db0a7ed4d35e6d0fa/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:55.086141 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30523e26-d9ea-47ac-b73a-31a2fe46ebac admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac78a09616cf46e5a4cd6297ed41ad88, user_id=87ecdefbce014a9db0a7ed4d35e6d0fa, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:55.088807 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30523e26-d9ea-47ac-b73a-31a2fe46ebac admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:55.149684 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1388/2793] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:58:55 2022] PUT /identity/v3/projects/ac78a09616cf46e5a4cd6297ed41ad88/users/87ecdefbce014a9db0a7ed4d35e6d0fa/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:55.163047 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9338aa3-b6a4-47c6-8f20-12cec196e8b4 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:55.163310 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9338aa3-b6a4-47c6-8f20-12cec196e8b4 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:55.163603 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9338aa3-b6a4-47c6-8f20-12cec196e8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:55.205529 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-b9338aa3-b6a4-47c6-8f20-12cec196e8b4 None None] MFA Rules not processed for user `87ecdefbce014a9db0a7ed4d35e6d0fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:58:55.206763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-b9338aa3-b6a4-47c6-8f20-12cec196e8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:55.285583 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1406/2794] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:58:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:58:55.297785 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30523e26-d9ea-47ac-b73a-31a2fe46ebac admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:55.299562 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-30523e26-d9ea-47ac-b73a-31a2fe46ebac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:55.354206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:55.360699 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30523e26-d9ea-47ac-b73a-31a2fe46ebac admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:55.364484 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc278f91-8a1f-4ff9-bf9e-429e312c13c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:55.365104 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc278f91-8a1f-4ff9-bf9e-429e312c13c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:55.365387 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc278f91-8a1f-4ff9-bf9e-429e312c13c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:55.365738 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc278f91-8a1f-4ff9-bf9e-429e312c13c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:55.376138 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc278f91-8a1f-4ff9-bf9e-429e312c13c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:55.377705 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc278f91-8a1f-4ff9-bf9e-429e312c13c3 service cinder] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:55.386920 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1389/2795] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:58:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:58:55.411988 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9338aa3-b6a4-47c6-8f20-12cec196e8b4 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:55.472895 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:55.484106 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9338aa3-b6a4-47c6-8f20-12cec196e8b4 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:55.484106 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feababf5-d131-4130-befb-e19d84c3a1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:55.486915 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feababf5-d131-4130-befb-e19d84c3a1af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:55.486915 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feababf5-d131-4130-befb-e19d84c3a1af admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:55.486915 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feababf5-d131-4130-befb-e19d84c3a1af admin admin] PATH_INFO: `/v3/users/87ecdefbce014a9db0a7ed4d35e6d0fa` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:55.486915 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feababf5-d131-4130-befb-e19d84c3a1af admin admin] RBAC: Authorizing `identity:delete_user(user_id=87ecdefbce014a9db0a7ed4d35e6d0fa)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:55.487870 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feababf5-d131-4130-befb-e19d84c3a1af admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:55.648619 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-feababf5-d131-4130-befb-e19d84c3a1af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87ecdefbce014a9db0a7ed4d35e6d0fa'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:55.656652 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-feababf5-d131-4130-befb-e19d84c3a1af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87ecdefbce014a9db0a7ed4d35e6d0fa'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:55.668420 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-feababf5-d131-4130-befb-e19d84c3a1af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87ecdefbce014a9db0a7ed4d35e6d0fa'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:55.672204 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-feababf5-d131-4130-befb-e19d84c3a1af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87ecdefbce014a9db0a7ed4d35e6d0fa'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:55.709452 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-feababf5-d131-4130-befb-e19d84c3a1af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87ecdefbce014a9db0a7ed4d35e6d0fa'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:55.716171 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1407/2796] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 12:58:55 2022] DELETE /identity/v3/users/87ecdefbce014a9db0a7ed4d35e6d0fa => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:58:55.720990 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc1be0-db25-42c3-9b62-712e9b8a0480 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:55.724031 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc1be0-db25-42c3-9b62-712e9b8a0480 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:55.724031 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc1be0-db25-42c3-9b62-712e9b8a0480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:55.741899 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-80dc1be0-db25-42c3-9b62-712e9b8a0480 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:58:55.743400 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-80dc1be0-db25-42c3-9b62-712e9b8a0480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:55.812199 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1390/2797] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:58:55.821883 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feababf5-d131-4130-befb-e19d84c3a1af admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:55.823551 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-feababf5-d131-4130-befb-e19d84c3a1af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:55.882527 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feababf5-d131-4130-befb-e19d84c3a1af admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:55.886142 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3168678c-b9ce-4d51-9cec-5c2fff4c4dfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:55.886754 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168678c-b9ce-4d51-9cec-5c2fff4c4dfb service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:55.887036 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168678c-b9ce-4d51-9cec-5c2fff4c4dfb service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:55.887840 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168678c-b9ce-4d51-9cec-5c2fff4c4dfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:55.897199 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3168678c-b9ce-4d51-9cec-5c2fff4c4dfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:55.898628 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3168678c-b9ce-4d51-9cec-5c2fff4c4dfb service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:55.909045 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1408/2798] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:58:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:58:56.316973 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc1be0-db25-42c3-9b62-712e9b8a0480 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:56.322432 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-80dc1be0-db25-42c3-9b62-712e9b8a0480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:56.330960 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc1be0-db25-42c3-9b62-712e9b8a0480 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:56.335773 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34cfb4f-5baf-43c8-96b5-1e511fd43b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:56.336415 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34cfb4f-5baf-43c8-96b5-1e511fd43b82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:56.336861 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34cfb4f-5baf-43c8-96b5-1e511fd43b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:56.337085 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34cfb4f-5baf-43c8-96b5-1e511fd43b82 admin admin] PATH_INFO: `/v3/projects/ac78a09616cf46e5a4cd6297ed41ad88` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:56.338366 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34cfb4f-5baf-43c8-96b5-1e511fd43b82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac78a09616cf46e5a4cd6297ed41ad88)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:56.340342 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34cfb4f-5baf-43c8-96b5-1e511fd43b82 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:56.437148 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a34cfb4f-5baf-43c8-96b5-1e511fd43b82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac78a09616cf46e5a4cd6297ed41ad88', 'request_id': 'req-a34cfb4f-5baf-43c8-96b5-1e511fd43b82'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:56.443134 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a34cfb4f-5baf-43c8-96b5-1e511fd43b82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac78a09616cf46e5a4cd6297ed41ad88', 'request_id': 'req-a34cfb4f-5baf-43c8-96b5-1e511fd43b82'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:56.521181 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a34cfb4f-5baf-43c8-96b5-1e511fd43b82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac78a09616cf46e5a4cd6297ed41ad88', 'request_id': 'req-a34cfb4f-5baf-43c8-96b5-1e511fd43b82'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:56.534736 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a34cfb4f-5baf-43c8-96b5-1e511fd43b82 admin admin] The token cache is being invalidate because project ac78a09616cf46e5a4cd6297ed41ad88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 12:58:56.534973 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a34cfb4f-5baf-43c8-96b5-1e511fd43b82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:56.535650 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-a34cfb4f-5baf-43c8-96b5-1e511fd43b82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 12:58:56.537269 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1391/2799] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 12:58:56 2022] DELETE /identity/v3/projects/ac78a09616cf46e5a4cd6297ed41ad88 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:58:56.542200 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f5e48-a779-4fa9-b06b-25aee3b67678 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:56.542493 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f5e48-a779-4fa9-b06b-25aee3b67678 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:56.542791 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f5e48-a779-4fa9-b06b-25aee3b67678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:56.563811 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-ac8f5e48-a779-4fa9-b06b-25aee3b67678 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:58:56.565009 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-ac8f5e48-a779-4fa9-b06b-25aee3b67678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:56.626631 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1409/2800] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:58:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:58:56.634322 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145be550-9700-4271-b1a7-10d427d8d58d None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:56.638458 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145be550-9700-4271-b1a7-10d427d8d58d None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:56.638458 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145be550-9700-4271-b1a7-10d427d8d58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:56.656427 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-145be550-9700-4271-b1a7-10d427d8d58d None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:58:56.661899 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-145be550-9700-4271-b1a7-10d427d8d58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:56.665949 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1392/2801] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:58:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:58:56.670207 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8f5e48-a779-4fa9-b06b-25aee3b67678 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:56.680251 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8f5e48-a779-4fa9-b06b-25aee3b67678 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:56.685006 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8335bfc2-925e-46b0-9d71-466f85ca1a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:56.685537 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335bfc2-925e-46b0-9d71-466f85ca1a7b admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:56.685817 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335bfc2-925e-46b0-9d71-466f85ca1a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:56.686089 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335bfc2-925e-46b0-9d71-466f85ca1a7b admin admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:56.686739 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8335bfc2-925e-46b0-9d71-466f85ca1a7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 12:58:56.687052 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8335bfc2-925e-46b0-9d71-466f85ca1a7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:56.693362 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8335bfc2-925e-46b0-9d71-466f85ca1a7b admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:56.701300 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1410/2802] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 12:58:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 12:58:56.705732 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145be550-9700-4271-b1a7-10d427d8d58d None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:56.716936 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145be550-9700-4271-b1a7-10d427d8d58d None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:56.721725 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66200392-3bdb-4752-a750-06af1b233322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:56.722377 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66200392-3bdb-4752-a750-06af1b233322 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:56.722662 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66200392-3bdb-4752-a750-06af1b233322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:56.722937 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66200392-3bdb-4752-a750-06af1b233322 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:56.723642 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66200392-3bdb-4752-a750-06af1b233322 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:56.725652 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66200392-3bdb-4752-a750-06af1b233322 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:56.817526 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1393/2803] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:58:56 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:58:56.818374 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8335bfc2-925e-46b0-9d71-466f85ca1a7b admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:56.825323 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:56.829565 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8335bfc2-925e-46b0-9d71-466f85ca1a7b admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:56.895877 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4e8d35-d722-4b23-b99c-96d1c5f32b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:56.896525 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e8d35-d722-4b23-b99c-96d1c5f32b20 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:56.896831 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e8d35-d722-4b23-b99c-96d1c5f32b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:56.897142 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e8d35-d722-4b23-b99c-96d1c5f32b20 admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:56.897724 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4e8d35-d722-4b23-b99c-96d1c5f32b20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:56.899661 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4e8d35-d722-4b23-b99c-96d1c5f32b20 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:57.014022 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1411/2804] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:58:56 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:58:57.018302 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66200392-3bdb-4752-a750-06af1b233322 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:57.029288 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66200392-3bdb-4752-a750-06af1b233322 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:57.033847 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc0e5b-eeb6-4a4f-a1bc-1e9d7e692e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:57.034454 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc0e5b-eeb6-4a4f-a1bc-1e9d7e692e78 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:57.034731 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc0e5b-eeb6-4a4f-a1bc-1e9d7e692e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:57.035010 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc0e5b-eeb6-4a4f-a1bc-1e9d7e692e78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:57.035684 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffc0e5b-eeb6-4a4f-a1bc-1e9d7e692e78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:57.037301 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffc0e5b-eeb6-4a4f-a1bc-1e9d7e692e78 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:57.047555 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1394/2805] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:58:57.051489 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4e8d35-d722-4b23-b99c-96d1c5f32b20 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:57.062260 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4e8d35-d722-4b23-b99c-96d1c5f32b20 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:57.066953 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c3c4c-8e0a-40b5-bc60-4673aec9429c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:57.067699 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c3c4c-8e0a-40b5-bc60-4673aec9429c admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:57.068043 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c3c4c-8e0a-40b5-bc60-4673aec9429c admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:57.068381 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c3c4c-8e0a-40b5-bc60-4673aec9429c admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:57.069085 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c3c4c-8e0a-40b5-bc60-4673aec9429c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:57.070766 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c3c4c-8e0a-40b5-bc60-4673aec9429c admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:57.082208 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1412/2806] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:58:57.086514 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc0e5b-eeb6-4a4f-a1bc-1e9d7e692e78 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:57.096886 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc0e5b-eeb6-4a4f-a1bc-1e9d7e692e78 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:57.101448 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13878aa3-caa7-4c68-8b8d-e33794117fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:57.102098 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13878aa3-caa7-4c68-8b8d-e33794117fc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:57.102375 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13878aa3-caa7-4c68-8b8d-e33794117fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:57.102697 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13878aa3-caa7-4c68-8b8d-e33794117fc5 admin admin] PATH_INFO: `/v3/projects/a0a4271779a04986bbd34e2e989f3603/users/9a4a9a635aaf44c7b81224a6b0ad9dec/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:57.117861 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13878aa3-caa7-4c68-8b8d-e33794117fc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0a4271779a04986bbd34e2e989f3603, user_id=9a4a9a635aaf44c7b81224a6b0ad9dec, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:57.120541 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13878aa3-caa7-4c68-8b8d-e33794117fc5 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:57.239628 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1395/2807] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:58:57 2022] PUT /identity/v3/projects/a0a4271779a04986bbd34e2e989f3603/users/9a4a9a635aaf44c7b81224a6b0ad9dec/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 12:58:57.248670 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf435d3-fb96-4ddb-9101-4f2f128d9829 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:57.248670 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf435d3-fb96-4ddb-9101-4f2f128d9829 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:57.248670 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf435d3-fb96-4ddb-9101-4f2f128d9829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:57.269030 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-bdf435d3-fb96-4ddb-9101-4f2f128d9829 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:58:57.270147 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-bdf435d3-fb96-4ddb-9101-4f2f128d9829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:57.330809 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1413/2808] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 12:58:57.342596 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13878aa3-caa7-4c68-8b8d-e33794117fc5 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:58:57.344571 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-13878aa3-caa7-4c68-8b8d-e33794117fc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:58:57.355886 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:58:57.403933 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13878aa3-caa7-4c68-8b8d-e33794117fc5 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:58:57.407585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542be643-c6a7-4a2e-b6e2-53ce7347621e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:58:57.408193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542be643-c6a7-4a2e-b6e2-53ce7347621e service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:58:57.408472 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542be643-c6a7-4a2e-b6e2-53ce7347621e service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:58:57.408743 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542be643-c6a7-4a2e-b6e2-53ce7347621e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:58:57.417883 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542be643-c6a7-4a2e-b6e2-53ce7347621e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:58:57.420639 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542be643-c6a7-4a2e-b6e2-53ce7347621e service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:58:57.430661 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1396/2809] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:58:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:59:03.025357 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68a594-f5da-4566-8c7f-5f446ebb24ad None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:59:03.025916 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68a594-f5da-4566-8c7f-5f446ebb24ad None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:59:03.026073 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68a594-f5da-4566-8c7f-5f446ebb24ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:59:03.064255 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-bc68a594-f5da-4566-8c7f-5f446ebb24ad None None] MFA Rules not processed for user `9a4a9a635aaf44c7b81224a6b0ad9dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:59:03.069165 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-bc68a594-f5da-4566-8c7f-5f446ebb24ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:59:03.145108 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1414/2810] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:59:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:59:03.149607 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542be643-c6a7-4a2e-b6e2-53ce7347621e service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:59:03.160884 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542be643-c6a7-4a2e-b6e2-53ce7347621e service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:59:03.165583 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f7734-841a-4812-9997-da69ce81b6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:59:03.166252 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f7734-841a-4812-9997-da69ce81b6bb admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:59:03.166484 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f7734-841a-4812-9997-da69ce81b6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:59:03.166756 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f7734-841a-4812-9997-da69ce81b6bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:59:03.167619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7f7734-841a-4812-9997-da69ce81b6bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:59:03.170780 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7f7734-841a-4812-9997-da69ce81b6bb admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:59:03.269107 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1397/2811] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 12:59:03 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 12:59:03.273184 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68a594-f5da-4566-8c7f-5f446ebb24ad None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:59:03.288308 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68a594-f5da-4566-8c7f-5f446ebb24ad None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:59:03.300844 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:59:03.358216 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c97df4d-458b-4cd2-b573-991e944924fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:59:03.358984 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97df4d-458b-4cd2-b573-991e944924fb admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:59:03.359402 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97df4d-458b-4cd2-b573-991e944924fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:59:03.359787 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97df4d-458b-4cd2-b573-991e944924fb admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:59:03.360540 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c97df4d-458b-4cd2-b573-991e944924fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:59:03.362575 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c97df4d-458b-4cd2-b573-991e944924fb admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:59:03.479354 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1415/2812] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 12:59:03 2022] POST /identity/v3/users => generated 298 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 12:59:03.480829 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f7734-841a-4812-9997-da69ce81b6bb admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:59:03.494020 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f7734-841a-4812-9997-da69ce81b6bb admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:59:03.499189 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e22272-c746-40d1-84e8-e931b1745e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:59:03.499887 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e22272-c746-40d1-84e8-e931b1745e22 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:59:03.500165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e22272-c746-40d1-84e8-e931b1745e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:59:03.500434 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e22272-c746-40d1-84e8-e931b1745e22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:59:03.501078 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e22272-c746-40d1-84e8-e931b1745e22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:59:03.502741 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e22272-c746-40d1-84e8-e931b1745e22 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:59:03.515636 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1398/2813] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:59:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:59:03.519687 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c97df4d-458b-4cd2-b573-991e944924fb admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:59:03.530322 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c97df4d-458b-4cd2-b573-991e944924fb admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:59:03.535308 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de020649-a286-46f5-8e69-85f02b35dc2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:59:03.536012 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de020649-a286-46f5-8e69-85f02b35dc2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:59:03.536292 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de020649-a286-46f5-8e69-85f02b35dc2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:59:03.536565 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de020649-a286-46f5-8e69-85f02b35dc2a admin admin] PATH_INFO: `/v3/projects/a7879e312b194158b24bdba87ab811c9/users/5502fe40e74f461aa1bbc6f563dc5654/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:59:03.553045 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de020649-a286-46f5-8e69-85f02b35dc2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7879e312b194158b24bdba87ab811c9, user_id=5502fe40e74f461aa1bbc6f563dc5654, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:59:03.556021 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de020649-a286-46f5-8e69-85f02b35dc2a admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:59:03.637892 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1416/2814] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:59:03 2022] PUT /identity/v3/projects/a7879e312b194158b24bdba87ab811c9/users/5502fe40e74f461aa1bbc6f563dc5654/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:59:03.641834 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e22272-c746-40d1-84e8-e931b1745e22 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:59:03.652128 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e22272-c746-40d1-84e8-e931b1745e22 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:59:03.710210 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4751852-d914-4ea2-b26b-a8e595a64042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:59:03.710735 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4751852-d914-4ea2-b26b-a8e595a64042 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:59:03.711013 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4751852-d914-4ea2-b26b-a8e595a64042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:59:03.711280 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4751852-d914-4ea2-b26b-a8e595a64042 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:59:03.711947 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4751852-d914-4ea2-b26b-a8e595a64042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:59:03.713526 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4751852-d914-4ea2-b26b-a8e595a64042 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:59:03.723668 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1399/2815] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 12:59:03 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 12:59:03.728113 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de020649-a286-46f5-8e69-85f02b35dc2a admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:59:03.739016 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de020649-a286-46f5-8e69-85f02b35dc2a admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:59:03.743971 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851963e-c2cc-4fb3-97a8-4ddea95c1e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:59:03.744625 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851963e-c2cc-4fb3-97a8-4ddea95c1e42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:59:03.744909 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851963e-c2cc-4fb3-97a8-4ddea95c1e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:59:03.745187 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851963e-c2cc-4fb3-97a8-4ddea95c1e42 admin admin] PATH_INFO: `/v3/projects/a7879e312b194158b24bdba87ab811c9/users/5502fe40e74f461aa1bbc6f563dc5654/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:59:03.747900 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851963e-c2cc-4fb3-97a8-4ddea95c1e42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7879e312b194158b24bdba87ab811c9, user_id=5502fe40e74f461aa1bbc6f563dc5654, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:59:03.750529 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851963e-c2cc-4fb3-97a8-4ddea95c1e42 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:59:03.760981 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1417/2816] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 12:59:03 2022] PUT /identity/v3/projects/a7879e312b194158b24bdba87ab811c9/users/5502fe40e74f461aa1bbc6f563dc5654/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 12:59:09.156973 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63028196-3e34-4d41-ba2e-e888c66b861c None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:59:09.157545 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63028196-3e34-4d41-ba2e-e888c66b861c None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:59:09.157545 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63028196-3e34-4d41-ba2e-e888c66b861c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:59:09.190474 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-63028196-3e34-4d41-ba2e-e888c66b861c None None] MFA Rules not processed for user `5502fe40e74f461aa1bbc6f563dc5654`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:59:09.192721 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-63028196-3e34-4d41-ba2e-e888c66b861c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:59:09.279176 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:59:09.283948 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1400/2817] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:59:09 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:59:09.297245 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851963e-c2cc-4fb3-97a8-4ddea95c1e42 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:59:09.301227 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-8851963e-c2cc-4fb3-97a8-4ddea95c1e42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:59:09.404221 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851963e-c2cc-4fb3-97a8-4ddea95c1e42 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:59:09.408427 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976d3f3-fd9b-4693-bfa7-660b73348fe9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b697e4847b13496d98a543675f5df1c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:59:09.409168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976d3f3-fd9b-4693-bfa7-660b73348fe9 service glance] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:59:09.409514 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976d3f3-fd9b-4693-bfa7-660b73348fe9 service glance] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:59:09.409864 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976d3f3-fd9b-4693-bfa7-660b73348fe9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:59:09.480131 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d976d3f3-fd9b-4693-bfa7-660b73348fe9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:59:09.482206 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d976d3f3-fd9b-4693-bfa7-660b73348fe9 service glance] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:59:09.494159 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1418/2818] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:59:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:59:09.799961 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c000caa-ad7b-453a-af17-74fe2c605c73 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:59:09.800248 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c000caa-ad7b-453a-af17-74fe2c605c73 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:59:09.800550 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c000caa-ad7b-453a-af17-74fe2c605c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:59:09.836146 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-2c000caa-ad7b-453a-af17-74fe2c605c73 None None] MFA Rules not processed for user `2e6e4387a7c24bbc82c22eefe2a3d138`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:59:09.838480 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-2c000caa-ad7b-453a-af17-74fe2c605c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:59:09.906607 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1401/2819] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Oct 3 12:59:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:59:10.045515 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976d3f3-fd9b-4693-bfa7-660b73348fe9 service glance] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:59:10.046935 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-d976d3f3-fd9b-4693-bfa7-660b73348fe9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:59:10.070342 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:59:10.122441 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976d3f3-fd9b-4693-bfa7-660b73348fe9 service glance] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:59:10.128237 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf2c86-ca91-465d-98a0-774d0ced76ef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b111d72a34144073917de4f566a58233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:59:10.128857 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf2c86-ca91-465d-98a0-774d0ced76ef service swift] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:59:10.129140 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf2c86-ca91-465d-98a0-774d0ced76ef service swift] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:59:10.129413 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf2c86-ca91-465d-98a0-774d0ced76ef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:59:10.139179 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bf2c86-ca91-465d-98a0-774d0ced76ef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:59:10.140809 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bf2c86-ca91-465d-98a0-774d0ced76ef service swift] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:59:10.149506 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1419/2820] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Oct 3 12:59:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 12:59:19.898618 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=32 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:59:20.146060 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=23 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:59:20.776352 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a093e0e-7cb7-45bd-9fea-f51794b9fefb None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:59:20.776704 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a093e0e-7cb7-45bd-9fea-f51794b9fefb None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:59:20.776909 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a093e0e-7cb7-45bd-9fea-f51794b9fefb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:59:20.797165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-7a093e0e-7cb7-45bd-9fea-f51794b9fefb None None] MFA Rules not processed for user `9a4a9a635aaf44c7b81224a6b0ad9dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:59:20.798366 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-7a093e0e-7cb7-45bd-9fea-f51794b9fefb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:59:20.805314 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1402/2821] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:59:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:59:24.472971 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36eda09-3d49-48b6-9495-32ec3cfd2386 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:59:24.473513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36eda09-3d49-48b6-9495-32ec3cfd2386 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:59:24.473513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36eda09-3d49-48b6-9495-32ec3cfd2386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:59:24.494205 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-e36eda09-3d49-48b6-9495-32ec3cfd2386 None None] MFA Rules not processed for user `2e6e4387a7c24bbc82c22eefe2a3d138`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:59:24.495429 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-e36eda09-3d49-48b6-9495-32ec3cfd2386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:59:24.503097 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1420/2822] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Oct 3 12:59:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:59:24.517175 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a093e0e-7cb7-45bd-9fea-f51794b9fefb None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:59:24.529849 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a093e0e-7cb7-45bd-9fea-f51794b9fefb None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:59:24.534807 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e17467-6b84-4762-aba1-8a6d7963cdac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b111d72a34144073917de4f566a58233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:59:24.535442 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e17467-6b84-4762-aba1-8a6d7963cdac service swift] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:59:24.535726 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e17467-6b84-4762-aba1-8a6d7963cdac service swift] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:59:24.535999 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e17467-6b84-4762-aba1-8a6d7963cdac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:59:24.544295 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e17467-6b84-4762-aba1-8a6d7963cdac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:59:24.545874 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e17467-6b84-4762-aba1-8a6d7963cdac service swift] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:59:24.553983 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1403/2823] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Oct 3 12:59:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 12:59:27.125787 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36eda09-3d49-48b6-9495-32ec3cfd2386 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:59:27.127479 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-e36eda09-3d49-48b6-9495-32ec3cfd2386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:59:27.190919 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36eda09-3d49-48b6-9495-32ec3cfd2386 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:59:27.194836 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5358269-f314-437e-a36f-ee59e8404e6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:59:27.195366 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5358269-f314-437e-a36f-ee59e8404e6e service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:59:27.195620 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5358269-f314-437e-a36f-ee59e8404e6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:59:27.195891 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5358269-f314-437e-a36f-ee59e8404e6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:59:27.197804 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-c5358269-f314-437e-a36f-ee59e8404e6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:59:27.307677 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5358269-f314-437e-a36f-ee59e8404e6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:59:27.309966 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5358269-f314-437e-a36f-ee59e8404e6e service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:59:27.332074 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1421/2824] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:59:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 12:59:28.523746 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e17467-6b84-4762-aba1-8a6d7963cdac service swift] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:59:28.534859 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e17467-6b84-4762-aba1-8a6d7963cdac service swift] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:59:28.538994 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c840487a-158e-4f6a-be07-90c39159abb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:59:28.539631 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840487a-158e-4f6a-be07-90c39159abb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:59:28.539905 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840487a-158e-4f6a-be07-90c39159abb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:59:28.540170 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840487a-158e-4f6a-be07-90c39159abb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:59:28.541903 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-c840487a-158e-4f6a-be07-90c39159abb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:59:28.556856 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c840487a-158e-4f6a-be07-90c39159abb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:59:28.558098 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c840487a-158e-4f6a-be07-90c39159abb8 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:59:28.570423 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1404/2825] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:59:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 12:59:37.329070 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=66 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:59:38.564900 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=15 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 12:59:55.967682 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff24a89-fd23-4d85-82ca-f43d58d40cf7 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:59:55.968177 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff24a89-fd23-4d85-82ca-f43d58d40cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:59:55.968297 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff24a89-fd23-4d85-82ca-f43d58d40cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:59:55.987660 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-4ff24a89-fd23-4d85-82ca-f43d58d40cf7 None None] MFA Rules not processed for user `5e29a4c0276247c286cff3eed5a1253a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 12:59:55.988846 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-4ff24a89-fd23-4d85-82ca-f43d58d40cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 12:59:56.043656 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1422/2826] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 12:59:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 12:59:57.022223 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c840487a-158e-4f6a-be07-90c39159abb8 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 12:59:57.031084 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c840487a-158e-4f6a-be07-90c39159abb8 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 12:59:57.034921 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e74c03-4a7b-4079-b99b-5a53c6a55f41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 12:59:57.035586 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e74c03-4a7b-4079-b99b-5a53c6a55f41 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 12:59:57.035863 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e74c03-4a7b-4079-b99b-5a53c6a55f41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 12:59:57.036206 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e74c03-4a7b-4079-b99b-5a53c6a55f41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 12:59:57.046154 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e74c03-4a7b-4079-b99b-5a53c6a55f41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 12:59:57.047712 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e74c03-4a7b-4079-b99b-5a53c6a55f41 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 12:59:57.057964 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1405/2827] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 12:59:57 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:00:01.209839 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff24a89-fd23-4d85-82ca-f43d58d40cf7 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:00:01.211470 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-4ff24a89-fd23-4d85-82ca-f43d58d40cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:00:01.228465 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff24a89-fd23-4d85-82ca-f43d58d40cf7 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:00:01.240051 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a01e8be-09e5-4db5-b944-0fe7e7f5592a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:00:01.240778 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a01e8be-09e5-4db5-b944-0fe7e7f5592a service nova] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:00:01.241062 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a01e8be-09e5-4db5-b944-0fe7e7f5592a service nova] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:00:01.241357 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a01e8be-09e5-4db5-b944-0fe7e7f5592a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:00:01.243394 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-9a01e8be-09e5-4db5-b944-0fe7e7f5592a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:00:01.260058 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a01e8be-09e5-4db5-b944-0fe7e7f5592a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:00:01.261749 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a01e8be-09e5-4db5-b944-0fe7e7f5592a service nova] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:00:01.273908 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1423/2828] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 13:00:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:00:06.885660 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e74c03-4a7b-4079-b99b-5a53c6a55f41 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:00:06.895725 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e74c03-4a7b-4079-b99b-5a53c6a55f41 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:00:06.899561 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7be878-f5d9-46be-8ff5-20cff6fb0ba1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:00:06.902852 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7be878-f5d9-46be-8ff5-20cff6fb0ba1 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:00:06.902852 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7be878-f5d9-46be-8ff5-20cff6fb0ba1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:00:06.902852 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7be878-f5d9-46be-8ff5-20cff6fb0ba1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:00:06.902852 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-bb7be878-f5d9-46be-8ff5-20cff6fb0ba1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:00:06.966056 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7be878-f5d9-46be-8ff5-20cff6fb0ba1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:00:06.967527 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7be878-f5d9-46be-8ff5-20cff6fb0ba1 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:00:06.977165 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1406/2829] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 13:00:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:00:11.269435 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=31 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:00:14.595728 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a01e8be-09e5-4db5-b944-0fe7e7f5592a service nova] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:00:14.597335 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-9a01e8be-09e5-4db5-b944-0fe7e7f5592a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:00:14.612100 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a01e8be-09e5-4db5-b944-0fe7e7f5592a service nova] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:00:14.616869 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f7b92-a3b4-453c-a08c-f2f1f2925e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:00:14.617499 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f7b92-a3b4-453c-a08c-f2f1f2925e78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:00:14.617777 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f7b92-a3b4-453c-a08c-f2f1f2925e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:00:14.618048 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f7b92-a3b4-453c-a08c-f2f1f2925e78 admin admin] PATH_INFO: `/v3/users/7ffffa27986f461ebc10c23c3c9817f0` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:00:14.619444 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f7b92-a3b4-453c-a08c-f2f1f2925e78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ffffa27986f461ebc10c23c3c9817f0)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:00:14.621339 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f7b92-a3b4-453c-a08c-f2f1f2925e78 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:00:14.819571 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-e08f7b92-a3b4-453c-a08c-f2f1f2925e78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ffffa27986f461ebc10c23c3c9817f0'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:00:14.826512 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-e08f7b92-a3b4-453c-a08c-f2f1f2925e78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ffffa27986f461ebc10c23c3c9817f0'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:00:14.838307 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-e08f7b92-a3b4-453c-a08c-f2f1f2925e78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ffffa27986f461ebc10c23c3c9817f0'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:00:14.838997 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-e08f7b92-a3b4-453c-a08c-f2f1f2925e78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ffffa27986f461ebc10c23c3c9817f0'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:00:14.909097 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-e08f7b92-a3b4-453c-a08c-f2f1f2925e78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ffffa27986f461ebc10c23c3c9817f0'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:00:14.915105 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1424/2830] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 13:00:14 2022] DELETE /identity/v3/users/7ffffa27986f461ebc10c23c3c9817f0 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:00:15.162633 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7be878-f5d9-46be-8ff5-20cff6fb0ba1 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:00:15.172741 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7be878-f5d9-46be-8ff5-20cff6fb0ba1 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:00:15.242623 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c9a974-bc78-4515-81f5-a249002a1b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:00:15.244503 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c9a974-bc78-4515-81f5-a249002a1b1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:00:15.244503 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c9a974-bc78-4515-81f5-a249002a1b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:00:15.244503 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c9a974-bc78-4515-81f5-a249002a1b1f admin admin] PATH_INFO: `/v3/projects/38d1778202d041b58934150c6af9685c` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:00:15.245750 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c9a974-bc78-4515-81f5-a249002a1b1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=38d1778202d041b58934150c6af9685c)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:00:15.247736 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c9a974-bc78-4515-81f5-a249002a1b1f admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:00:15.372698 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-18c9a974-bc78-4515-81f5-a249002a1b1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38d1778202d041b58934150c6af9685c', 'request_id': 'req-18c9a974-bc78-4515-81f5-a249002a1b1f'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:00:15.378089 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-18c9a974-bc78-4515-81f5-a249002a1b1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38d1778202d041b58934150c6af9685c', 'request_id': 'req-18c9a974-bc78-4515-81f5-a249002a1b1f'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:00:15.446397 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-18c9a974-bc78-4515-81f5-a249002a1b1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38d1778202d041b58934150c6af9685c', 'request_id': 'req-18c9a974-bc78-4515-81f5-a249002a1b1f'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:00:15.458896 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-18c9a974-bc78-4515-81f5-a249002a1b1f admin admin] The token cache is being invalidate because project 38d1778202d041b58934150c6af9685c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:00:15.459187 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-18c9a974-bc78-4515-81f5-a249002a1b1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:00:15.459840 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-18c9a974-bc78-4515-81f5-a249002a1b1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:00:15.461360 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1407/2831] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 13:00:15 2022] DELETE /identity/v3/projects/38d1778202d041b58934150c6af9685c => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:00:15.465494 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f7b92-a3b4-453c-a08c-f2f1f2925e78 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:00:15.466916 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-e08f7b92-a3b4-453c-a08c-f2f1f2925e78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:00:15.480987 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f7b92-a3b4-453c-a08c-f2f1f2925e78 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:00:15.537545 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48738041-2cea-4dde-a9a8-54bf65415f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:00:15.538176 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48738041-2cea-4dde-a9a8-54bf65415f28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:00:15.538453 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48738041-2cea-4dde-a9a8-54bf65415f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:00:15.538728 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48738041-2cea-4dde-a9a8-54bf65415f28 admin admin] PATH_INFO: `/v3/users/7a5d1b590faa430e8ebb9c7b0f91ec4d` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:00:15.539884 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48738041-2cea-4dde-a9a8-54bf65415f28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a5d1b590faa430e8ebb9c7b0f91ec4d)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:00:15.541780 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48738041-2cea-4dde-a9a8-54bf65415f28 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:00:15.717185 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-48738041-2cea-4dde-a9a8-54bf65415f28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a5d1b590faa430e8ebb9c7b0f91ec4d'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:00:15.724458 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-48738041-2cea-4dde-a9a8-54bf65415f28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a5d1b590faa430e8ebb9c7b0f91ec4d'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:00:15.736313 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-48738041-2cea-4dde-a9a8-54bf65415f28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a5d1b590faa430e8ebb9c7b0f91ec4d'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:00:15.736974 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-48738041-2cea-4dde-a9a8-54bf65415f28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a5d1b590faa430e8ebb9c7b0f91ec4d'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:00:15.738608 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:00:15.834341 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-48738041-2cea-4dde-a9a8-54bf65415f28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a5d1b590faa430e8ebb9c7b0f91ec4d'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:00:15.840419 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1425/2832] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 13:00:15 2022] DELETE /identity/v3/users/7a5d1b590faa430e8ebb9c7b0f91ec4d => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:00:16.087746 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c9a974-bc78-4515-81f5-a249002a1b1f admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:00:16.092034 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-18c9a974-bc78-4515-81f5-a249002a1b1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:00:16.104489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c9a974-bc78-4515-81f5-a249002a1b1f admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:00:16.114475 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:00:16.167477 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767b9612-02b6-4fc1-9bd4-7f08fa1469ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:00:16.168215 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b9612-02b6-4fc1-9bd4-7f08fa1469ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:00:16.168384 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b9612-02b6-4fc1-9bd4-7f08fa1469ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:00:16.168656 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b9612-02b6-4fc1-9bd4-7f08fa1469ff admin admin] PATH_INFO: `/v3/projects/255d6627821c48b38f53f9abe4b6e3ae` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:00:16.169766 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767b9612-02b6-4fc1-9bd4-7f08fa1469ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=255d6627821c48b38f53f9abe4b6e3ae)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:00:16.172154 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767b9612-02b6-4fc1-9bd4-7f08fa1469ff admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:00:16.339276 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-767b9612-02b6-4fc1-9bd4-7f08fa1469ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '255d6627821c48b38f53f9abe4b6e3ae', 'request_id': 'req-767b9612-02b6-4fc1-9bd4-7f08fa1469ff'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:00:16.344922 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-767b9612-02b6-4fc1-9bd4-7f08fa1469ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '255d6627821c48b38f53f9abe4b6e3ae', 'request_id': 'req-767b9612-02b6-4fc1-9bd4-7f08fa1469ff'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:00:16.422545 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-767b9612-02b6-4fc1-9bd4-7f08fa1469ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '255d6627821c48b38f53f9abe4b6e3ae', 'request_id': 'req-767b9612-02b6-4fc1-9bd4-7f08fa1469ff'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:00:16.434145 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-767b9612-02b6-4fc1-9bd4-7f08fa1469ff admin admin] The token cache is being invalidate because project 255d6627821c48b38f53f9abe4b6e3ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:00:16.434436 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-767b9612-02b6-4fc1-9bd4-7f08fa1469ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:00:16.435061 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-767b9612-02b6-4fc1-9bd4-7f08fa1469ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:00:16.436555 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1408/2833] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 13:00:16 2022] DELETE /identity/v3/projects/255d6627821c48b38f53f9abe4b6e3ae => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:00:25.914351 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:00:26.431175 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:00:45.905189 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48738041-2cea-4dde-a9a8-54bf65415f28 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:00:45.906619 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-48738041-2cea-4dde-a9a8-54bf65415f28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:00:45.963992 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48738041-2cea-4dde-a9a8-54bf65415f28 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:00:46.012943 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82bfb35-f2c4-47f7-bd18-07af11089367 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:00:46.013551 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82bfb35-f2c4-47f7-bd18-07af11089367 service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:00:46.013830 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82bfb35-f2c4-47f7-bd18-07af11089367 service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:00:46.014099 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82bfb35-f2c4-47f7-bd18-07af11089367 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:00:46.015910 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-c82bfb35-f2c4-47f7-bd18-07af11089367 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:00:46.025481 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82bfb35-f2c4-47f7-bd18-07af11089367 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:00:46.026941 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82bfb35-f2c4-47f7-bd18-07af11089367 service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:00:46.034866 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1426/2834] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 13:00:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:00:56.031219 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=46 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:01:08.221009 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767b9612-02b6-4fc1-9bd4-7f08fa1469ff admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:08.223305 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-767b9612-02b6-4fc1-9bd4-7f08fa1469ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:08.358814 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767b9612-02b6-4fc1-9bd4-7f08fa1469ff admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:08.366417 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18453c08-51df-492f-bdd6-5c8c62c6c1d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:08.367050 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18453c08-51df-492f-bdd6-5c8c62c6c1d7 service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:08.367415 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18453c08-51df-492f-bdd6-5c8c62c6c1d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:08.367748 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18453c08-51df-492f-bdd6-5c8c62c6c1d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:08.372022 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-18453c08-51df-492f-bdd6-5c8c62c6c1d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:08.387948 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18453c08-51df-492f-bdd6-5c8c62c6c1d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:08.389552 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18453c08-51df-492f-bdd6-5c8c62c6c1d7 service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:08.406397 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1409/2835] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 13:01:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:01:13.026199 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82bfb35-f2c4-47f7-bd18-07af11089367 service neutron] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:13.027689 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-c82bfb35-f2c4-47f7-bd18-07af11089367 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:13.100949 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82bfb35-f2c4-47f7-bd18-07af11089367 service neutron] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:13.106107 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501ed90-0fef-4a80-9617-42d613fd94a6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25eef64e92e9422ea62508c3eb0a197e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:13.106701 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501ed90-0fef-4a80-9617-42d613fd94a6 service placement] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:13.106976 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501ed90-0fef-4a80-9617-42d613fd94a6 service placement] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:13.107245 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501ed90-0fef-4a80-9617-42d613fd94a6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:13.108955 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-1501ed90-0fef-4a80-9617-42d613fd94a6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:13.120905 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1501ed90-0fef-4a80-9617-42d613fd94a6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:13.123008 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1501ed90-0fef-4a80-9617-42d613fd94a6 service placement] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:13.132740 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1427/2836] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 13:01:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:01:13.184643 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18453c08-51df-492f-bdd6-5c8c62c6c1d7 service nova] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:13.186170 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-18453c08-51df-492f-bdd6-5c8c62c6c1d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:13.254232 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18453c08-51df-492f-bdd6-5c8c62c6c1d7 service nova] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:13.258819 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e4a2f-dbce-456b-a6a4-75982cf55c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:13.259492 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e4a2f-dbce-456b-a6a4-75982cf55c7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:13.259778 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e4a2f-dbce-456b-a6a4-75982cf55c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:13.260055 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e4a2f-dbce-456b-a6a4-75982cf55c7b admin admin] PATH_INFO: `/v3/users/5e29a4c0276247c286cff3eed5a1253a` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:13.261287 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9e4a2f-dbce-456b-a6a4-75982cf55c7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e29a4c0276247c286cff3eed5a1253a)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:13.263207 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9e4a2f-dbce-456b-a6a4-75982cf55c7b admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:13.471535 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ec9e4a2f-dbce-456b-a6a4-75982cf55c7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e29a4c0276247c286cff3eed5a1253a'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:13.483479 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ec9e4a2f-dbce-456b-a6a4-75982cf55c7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e29a4c0276247c286cff3eed5a1253a'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:13.484190 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ec9e4a2f-dbce-456b-a6a4-75982cf55c7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e29a4c0276247c286cff3eed5a1253a'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:13.492425 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ec9e4a2f-dbce-456b-a6a4-75982cf55c7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e29a4c0276247c286cff3eed5a1253a'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:13.574801 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ec9e4a2f-dbce-456b-a6a4-75982cf55c7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e29a4c0276247c286cff3eed5a1253a'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:13.580930 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1410/2837] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 13:01:13 2022] DELETE /identity/v3/users/5e29a4c0276247c286cff3eed5a1253a => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:01:13.590860 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501ed90-0fef-4a80-9617-42d613fd94a6 service placement] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:13.592254 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-1501ed90-0fef-4a80-9617-42d613fd94a6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:13.604298 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501ed90-0fef-4a80-9617-42d613fd94a6 service placement] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:13.660606 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b416783-48ad-458a-89ff-e2a9308893c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:13.661235 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b416783-48ad-458a-89ff-e2a9308893c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:13.661523 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b416783-48ad-458a-89ff-e2a9308893c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:13.661804 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b416783-48ad-458a-89ff-e2a9308893c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:13.663578 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-7b416783-48ad-458a-89ff-e2a9308893c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:13.729351 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b416783-48ad-458a-89ff-e2a9308893c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:13.730776 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b416783-48ad-458a-89ff-e2a9308893c3 service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:13.740430 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1428/2838] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 13:01:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:01:14.089866 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e4a2f-dbce-456b-a6a4-75982cf55c7b admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:14.091306 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ec9e4a2f-dbce-456b-a6a4-75982cf55c7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:14.101513 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:01:14.105563 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e4a2f-dbce-456b-a6a4-75982cf55c7b admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:14.111574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec6da6-5c50-4782-b286-e89c6fe946b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:14.112291 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec6da6-5c50-4782-b286-e89c6fe946b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:14.113329 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec6da6-5c50-4782-b286-e89c6fe946b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:14.113609 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec6da6-5c50-4782-b286-e89c6fe946b8 admin admin] PATH_INFO: `/v3/projects/484522301976426dae50e64c141c727e` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:14.114891 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceec6da6-5c50-4782-b286-e89c6fe946b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=484522301976426dae50e64c141c727e)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:14.116848 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceec6da6-5c50-4782-b286-e89c6fe946b8 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:14.269658 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ceec6da6-5c50-4782-b286-e89c6fe946b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '484522301976426dae50e64c141c727e', 'request_id': 'req-ceec6da6-5c50-4782-b286-e89c6fe946b8'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:14.275120 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ceec6da6-5c50-4782-b286-e89c6fe946b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '484522301976426dae50e64c141c727e', 'request_id': 'req-ceec6da6-5c50-4782-b286-e89c6fe946b8'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:14.348697 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ceec6da6-5c50-4782-b286-e89c6fe946b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '484522301976426dae50e64c141c727e', 'request_id': 'req-ceec6da6-5c50-4782-b286-e89c6fe946b8'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:14.360073 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ceec6da6-5c50-4782-b286-e89c6fe946b8 admin admin] The token cache is being invalidate because project 484522301976426dae50e64c141c727e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:01:14.360366 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ceec6da6-5c50-4782-b286-e89c6fe946b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:14.361081 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-ceec6da6-5c50-4782-b286-e89c6fe946b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:14.362825 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1411/2839] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 13:01:14 2022] DELETE /identity/v3/projects/484522301976426dae50e64c141c727e => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:01:14.367989 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47666d66-4c50-4521-90b2-b7cbf4725551 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:14.370464 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47666d66-4c50-4521-90b2-b7cbf4725551 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:14.370464 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47666d66-4c50-4521-90b2-b7cbf4725551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:14.389066 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-47666d66-4c50-4521-90b2-b7cbf4725551 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:01:14.390262 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-47666d66-4c50-4521-90b2-b7cbf4725551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:14.423201 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=100 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:01:14.449536 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1429/2840] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 13:01:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:01:14.460279 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c212d-d2ce-46e0-bbb5-8b0a4f16864d None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:14.460678 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c212d-d2ce-46e0-bbb5-8b0a4f16864d None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:14.461060 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c212d-d2ce-46e0-bbb5-8b0a4f16864d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:14.481078 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-648c212d-d2ce-46e0-bbb5-8b0a4f16864d None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:01:14.482265 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-648c212d-d2ce-46e0-bbb5-8b0a4f16864d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:14.490621 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1412/2841] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 13:01:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:01:14.495045 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47666d66-4c50-4521-90b2-b7cbf4725551 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:14.505536 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47666d66-4c50-4521-90b2-b7cbf4725551 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:14.510011 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ec609-c356-46a8-b0c1-118099828dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:14.510613 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ec609-c356-46a8-b0c1-118099828dba admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:14.510893 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ec609-c356-46a8-b0c1-118099828dba admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:14.511168 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ec609-c356-46a8-b0c1-118099828dba admin admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:14.511843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62ec609-c356-46a8-b0c1-118099828dba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:01:14.512151 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62ec609-c356-46a8-b0c1-118099828dba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:14.513720 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62ec609-c356-46a8-b0c1-118099828dba admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:14.525475 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1430/2842] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 13:01:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:01:14.529587 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648c212d-d2ce-46e0-bbb5-8b0a4f16864d None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:14.538788 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648c212d-d2ce-46e0-bbb5-8b0a4f16864d None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:14.543274 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8a705d-58eb-4187-b478-458637cf35da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:14.543893 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a705d-58eb-4187-b478-458637cf35da admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:14.544171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a705d-58eb-4187-b478-458637cf35da admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:14.544442 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a705d-58eb-4187-b478-458637cf35da admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:14.545051 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8a705d-58eb-4187-b478-458637cf35da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:14.546927 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8a705d-58eb-4187-b478-458637cf35da admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:14.676367 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1413/2843] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 13:01:14 2022] POST /identity/v3/projects => generated 383 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:01:14.680513 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ec609-c356-46a8-b0c1-118099828dba admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:14.691669 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ec609-c356-46a8-b0c1-118099828dba admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:14.747712 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c58ab4c-60d1-4c09-aea4-a8d8e87ab445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:14.748322 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c58ab4c-60d1-4c09-aea4-a8d8e87ab445 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:14.748601 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c58ab4c-60d1-4c09-aea4-a8d8e87ab445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:14.748870 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c58ab4c-60d1-4c09-aea4-a8d8e87ab445 admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:14.749529 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c58ab4c-60d1-4c09-aea4-a8d8e87ab445 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:14.751495 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c58ab4c-60d1-4c09-aea4-a8d8e87ab445 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:14.840370 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1431/2844] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 13:01:14 2022] POST /identity/v3/users => generated 292 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:01:14.844238 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8a705d-58eb-4187-b478-458637cf35da admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:14.855024 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8a705d-58eb-4187-b478-458637cf35da admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:14.859640 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ebec4-6912-439e-8a3c-146c4894173d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:14.860240 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ebec4-6912-439e-8a3c-146c4894173d admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:14.860521 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ebec4-6912-439e-8a3c-146c4894173d admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:14.860797 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ebec4-6912-439e-8a3c-146c4894173d admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:14.861434 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781ebec4-6912-439e-8a3c-146c4894173d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:14.863632 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781ebec4-6912-439e-8a3c-146c4894173d admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:14.873859 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1414/2845] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 13:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:01:14.877856 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c58ab4c-60d1-4c09-aea4-a8d8e87ab445 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:14.887820 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c58ab4c-60d1-4c09-aea4-a8d8e87ab445 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:14.892356 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c24ee3-a159-460e-b086-f399e6ce123e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:14.892941 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c24ee3-a159-460e-b086-f399e6ce123e admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:14.893219 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c24ee3-a159-460e-b086-f399e6ce123e admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:14.893486 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c24ee3-a159-460e-b086-f399e6ce123e admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:14.894293 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c24ee3-a159-460e-b086-f399e6ce123e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:14.895896 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c24ee3-a159-460e-b086-f399e6ce123e admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:14.906141 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1432/2846] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 13:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:01:14.911109 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ebec4-6912-439e-8a3c-146c4894173d admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:14.921055 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ebec4-6912-439e-8a3c-146c4894173d admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:14.926846 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4fc6fb-761a-4419-82f2-bb6057e1da47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:14.927526 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fc6fb-761a-4419-82f2-bb6057e1da47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:14.927810 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fc6fb-761a-4419-82f2-bb6057e1da47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:14.928083 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fc6fb-761a-4419-82f2-bb6057e1da47 admin admin] PATH_INFO: `/v3/projects/013a17c83cf44302b0705de04214140e/users/d5a8888fcc784bdb9d5c6ebf7359dbc2/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:14.943286 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4fc6fb-761a-4419-82f2-bb6057e1da47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=013a17c83cf44302b0705de04214140e, user_id=d5a8888fcc784bdb9d5c6ebf7359dbc2, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:14.945989 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4fc6fb-761a-4419-82f2-bb6057e1da47 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:15.052244 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1415/2847] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 13:01:14 2022] PUT /identity/v3/projects/013a17c83cf44302b0705de04214140e/users/d5a8888fcc784bdb9d5c6ebf7359dbc2/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:01:15.062190 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0351785-83cd-4136-94eb-28643b16a328 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:15.062477 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0351785-83cd-4136-94eb-28643b16a328 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:15.062773 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0351785-83cd-4136-94eb-28643b16a328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:15.090255 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-a0351785-83cd-4136-94eb-28643b16a328 None None] MFA Rules not processed for user `d5a8888fcc784bdb9d5c6ebf7359dbc2`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:01:15.091546 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-a0351785-83cd-4136-94eb-28643b16a328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:15.158609 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1433/2848] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 13:01:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:01:15.169862 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4fc6fb-761a-4419-82f2-bb6057e1da47 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:15.171645 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-5a4fc6fb-761a-4419-82f2-bb6057e1da47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:15.252250 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4fc6fb-761a-4419-82f2-bb6057e1da47 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:15.258405 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfff87a-0a74-4883-a57d-505e36bd0275 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:15.259031 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfff87a-0a74-4883-a57d-505e36bd0275 service nova] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:15.259310 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfff87a-0a74-4883-a57d-505e36bd0275 service nova] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:15.259610 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfff87a-0a74-4883-a57d-505e36bd0275 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:15.267797 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfff87a-0a74-4883-a57d-505e36bd0275 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:15.269433 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfff87a-0a74-4883-a57d-505e36bd0275 service nova] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:15.274029 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:01:15.278868 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1416/2849] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 13:01:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:01:17.681238 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381aabd-8abe-4b88-8737-80e863191dc7 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:17.681726 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381aabd-8abe-4b88-8737-80e863191dc7 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:17.681946 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381aabd-8abe-4b88-8737-80e863191dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:17.734955 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-d381aabd-8abe-4b88-8737-80e863191dc7 None None] MFA Rules not processed for user `2e6e4387a7c24bbc82c22eefe2a3d138`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:01:17.736338 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-d381aabd-8abe-4b88-8737-80e863191dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:17.742799 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:01:17.803526 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1434/2850] 10.222.0.33 () {58 vars in 967 bytes} [Mon Oct 3 13:01:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:01:17.845085 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfff87a-0a74-4883-a57d-505e36bd0275 service nova] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:17.849584 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-1dfff87a-0a74-4883-a57d-505e36bd0275 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:17.943145 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfff87a-0a74-4883-a57d-505e36bd0275 service nova] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:17.947903 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bfc308-d474-4378-bf5e-7b8b78b11a4c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b111d72a34144073917de4f566a58233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:17.948507 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bfc308-d474-4378-bf5e-7b8b78b11a4c service swift] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:17.953016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bfc308-d474-4378-bf5e-7b8b78b11a4c service swift] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:17.953016 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bfc308-d474-4378-bf5e-7b8b78b11a4c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:17.956944 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bfc308-d474-4378-bf5e-7b8b78b11a4c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:17.959117 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bfc308-d474-4378-bf5e-7b8b78b11a4c service swift] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:17.967113 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1417/2851] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Oct 3 13:01:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 13:01:23.592091 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381aabd-8abe-4b88-8737-80e863191dc7 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:23.677497 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381aabd-8abe-4b88-8737-80e863191dc7 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:23.682527 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfaf8d7-e981-49fd-9b9f-8bed0cee0ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:23.683179 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfaf8d7-e981-49fd-9b9f-8bed0cee0ef2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:23.683509 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfaf8d7-e981-49fd-9b9f-8bed0cee0ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:23.683785 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfaf8d7-e981-49fd-9b9f-8bed0cee0ef2 admin admin] PATH_INFO: `/v3/users/d5a8888fcc784bdb9d5c6ebf7359dbc2` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:23.684990 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfaf8d7-e981-49fd-9b9f-8bed0cee0ef2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5a8888fcc784bdb9d5c6ebf7359dbc2)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:23.686975 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfaf8d7-e981-49fd-9b9f-8bed0cee0ef2 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:23.869549 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-fdfaf8d7-e981-49fd-9b9f-8bed0cee0ef2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5a8888fcc784bdb9d5c6ebf7359dbc2'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:23.876873 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-fdfaf8d7-e981-49fd-9b9f-8bed0cee0ef2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5a8888fcc784bdb9d5c6ebf7359dbc2'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:23.889431 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-fdfaf8d7-e981-49fd-9b9f-8bed0cee0ef2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5a8888fcc784bdb9d5c6ebf7359dbc2'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:23.890137 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-fdfaf8d7-e981-49fd-9b9f-8bed0cee0ef2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5a8888fcc784bdb9d5c6ebf7359dbc2'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:23.954479 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-fdfaf8d7-e981-49fd-9b9f-8bed0cee0ef2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5a8888fcc784bdb9d5c6ebf7359dbc2'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:23.961884 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1435/2852] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 13:01:23 2022] DELETE /identity/v3/users/d5a8888fcc784bdb9d5c6ebf7359dbc2 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:01:23.966443 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376352d2-162c-49bf-b5c9-25d3fe3902c1 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:23.966902 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376352d2-162c-49bf-b5c9-25d3fe3902c1 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:23.967030 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376352d2-162c-49bf-b5c9-25d3fe3902c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:23.986864 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-376352d2-162c-49bf-b5c9-25d3fe3902c1 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:01:23.988177 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-376352d2-162c-49bf-b5c9-25d3fe3902c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:24.049169 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1418/2853] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 13:01:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:01:24.058888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfaf8d7-e981-49fd-9b9f-8bed0cee0ef2 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:24.063896 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-fdfaf8d7-e981-49fd-9b9f-8bed0cee0ef2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:24.104377 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:01:24.135909 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfaf8d7-e981-49fd-9b9f-8bed0cee0ef2 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:24.139709 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f85404-1680-48c7-b803-0763bfda6021 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:24.140358 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f85404-1680-48c7-b803-0763bfda6021 service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:24.140678 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f85404-1680-48c7-b803-0763bfda6021 service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:24.140948 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f85404-1680-48c7-b803-0763bfda6021 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:24.151982 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f85404-1680-48c7-b803-0763bfda6021 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:24.155965 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f85404-1680-48c7-b803-0763bfda6021 service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:24.170768 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1436/2854] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 13:01:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:01:24.760873 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376352d2-162c-49bf-b5c9-25d3fe3902c1 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:24.760873 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-376352d2-162c-49bf-b5c9-25d3fe3902c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:24.770337 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376352d2-162c-49bf-b5c9-25d3fe3902c1 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:24.775098 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806ffe2b-c0ec-4ed2-9aba-acd9d7e8fa35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:24.775919 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ffe2b-c0ec-4ed2-9aba-acd9d7e8fa35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:24.776363 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ffe2b-c0ec-4ed2-9aba-acd9d7e8fa35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:24.776708 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ffe2b-c0ec-4ed2-9aba-acd9d7e8fa35 admin admin] PATH_INFO: `/v3/projects/013a17c83cf44302b0705de04214140e` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:24.778227 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806ffe2b-c0ec-4ed2-9aba-acd9d7e8fa35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=013a17c83cf44302b0705de04214140e)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:24.780804 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806ffe2b-c0ec-4ed2-9aba-acd9d7e8fa35 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:24.905505 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-806ffe2b-c0ec-4ed2-9aba-acd9d7e8fa35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '013a17c83cf44302b0705de04214140e', 'request_id': 'req-806ffe2b-c0ec-4ed2-9aba-acd9d7e8fa35'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:24.912656 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-806ffe2b-c0ec-4ed2-9aba-acd9d7e8fa35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '013a17c83cf44302b0705de04214140e', 'request_id': 'req-806ffe2b-c0ec-4ed2-9aba-acd9d7e8fa35'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:24.978236 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-806ffe2b-c0ec-4ed2-9aba-acd9d7e8fa35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '013a17c83cf44302b0705de04214140e', 'request_id': 'req-806ffe2b-c0ec-4ed2-9aba-acd9d7e8fa35'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:24.980043 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:01:24.991399 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-806ffe2b-c0ec-4ed2-9aba-acd9d7e8fa35 admin admin] The token cache is being invalidate because project 013a17c83cf44302b0705de04214140e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:01:24.991688 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-806ffe2b-c0ec-4ed2-9aba-acd9d7e8fa35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:24.993168 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-806ffe2b-c0ec-4ed2-9aba-acd9d7e8fa35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:24.994002 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1419/2855] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 13:01:24 2022] DELETE /identity/v3/projects/013a17c83cf44302b0705de04214140e => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:01:24.999283 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0e1b3-5e6e-4ebb-ac2e-c785a118ecaa None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:24.999851 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0e1b3-5e6e-4ebb-ac2e-c785a118ecaa None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:24.999953 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0e1b3-5e6e-4ebb-ac2e-c785a118ecaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:25.019941 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-22c0e1b3-5e6e-4ebb-ac2e-c785a118ecaa None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:01:25.021797 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-22c0e1b3-5e6e-4ebb-ac2e-c785a118ecaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:25.093905 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1437/2856] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 13:01:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:01:25.096805 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6dfe2-28d0-4d41-b033-3eed57483a12 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:25.097005 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6dfe2-28d0-4d41-b033-3eed57483a12 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:25.097832 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6dfe2-28d0-4d41-b033-3eed57483a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:25.120902 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-dfd6dfe2-28d0-4d41-b033-3eed57483a12 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:01:25.122108 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-dfd6dfe2-28d0-4d41-b033-3eed57483a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:25.130344 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1420/2857] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 13:01:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:01:25.134907 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0e1b3-5e6e-4ebb-ac2e-c785a118ecaa None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:25.144598 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0e1b3-5e6e-4ebb-ac2e-c785a118ecaa None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:25.150582 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0616c7fe-4b24-4515-a569-f1846a9e7637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:25.151234 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0616c7fe-4b24-4515-a569-f1846a9e7637 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:25.151547 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0616c7fe-4b24-4515-a569-f1846a9e7637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:25.151823 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0616c7fe-4b24-4515-a569-f1846a9e7637 admin admin] PATH_INFO: `/v3/domains` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:25.152550 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0616c7fe-4b24-4515-a569-f1846a9e7637 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81806) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:01:25.152870 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0616c7fe-4b24-4515-a569-f1846a9e7637 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:25.154666 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0616c7fe-4b24-4515-a569-f1846a9e7637 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:25.168432 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1438/2858] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 13:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:01:25.172729 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd6dfe2-28d0-4d41-b033-3eed57483a12 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:25.182538 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd6dfe2-28d0-4d41-b033-3eed57483a12 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:25.187249 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ab9ae4-182e-4c60-86e2-733d79a319c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:25.188045 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab9ae4-182e-4c60-86e2-733d79a319c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:25.188278 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab9ae4-182e-4c60-86e2-733d79a319c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:25.188548 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab9ae4-182e-4c60-86e2-733d79a319c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:25.189303 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ab9ae4-182e-4c60-86e2-733d79a319c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:25.191073 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ab9ae4-182e-4c60-86e2-733d79a319c9 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:25.299948 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1421/2859] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 13:01:25 2022] POST /identity/v3/projects => generated 405 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:01:25.301159 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0616c7fe-4b24-4515-a569-f1846a9e7637 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:25.311534 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0616c7fe-4b24-4515-a569-f1846a9e7637 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:25.372034 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b1009-7834-4a31-aff6-14eef0aac519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:25.372659 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b1009-7834-4a31-aff6-14eef0aac519 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:25.376768 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b1009-7834-4a31-aff6-14eef0aac519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:25.376768 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b1009-7834-4a31-aff6-14eef0aac519 admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:25.376768 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0b1009-7834-4a31-aff6-14eef0aac519 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:25.376768 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0b1009-7834-4a31-aff6-14eef0aac519 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:25.468940 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1439/2860] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 13:01:25 2022] POST /identity/v3/users => generated 303 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:01:25.472992 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ab9ae4-182e-4c60-86e2-733d79a319c9 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:25.484515 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ab9ae4-182e-4c60-86e2-733d79a319c9 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:25.489528 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a0705-c4cb-4c9d-838a-9cdab4b70800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:25.499655 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a0705-c4cb-4c9d-838a-9cdab4b70800 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:25.500257 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a0705-c4cb-4c9d-838a-9cdab4b70800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:25.500697 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a0705-c4cb-4c9d-838a-9cdab4b70800 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:25.501583 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a0705-c4cb-4c9d-838a-9cdab4b70800 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:25.503517 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a0705-c4cb-4c9d-838a-9cdab4b70800 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:25.522158 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1422/2861] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 13:01:25 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:01:25.527057 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b1009-7834-4a31-aff6-14eef0aac519 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:25.547901 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b1009-7834-4a31-aff6-14eef0aac519 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:25.553628 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b293b617-746d-4ebe-9ca9-eb28d4c111da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:25.554822 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b293b617-746d-4ebe-9ca9-eb28d4c111da admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:25.555107 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b293b617-746d-4ebe-9ca9-eb28d4c111da admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:25.555430 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b293b617-746d-4ebe-9ca9-eb28d4c111da admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:25.556336 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b293b617-746d-4ebe-9ca9-eb28d4c111da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:25.558001 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b293b617-746d-4ebe-9ca9-eb28d4c111da admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:25.574196 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1440/2862] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 13:01:25 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:01:25.578427 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a0705-c4cb-4c9d-838a-9cdab4b70800 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:25.593301 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a0705-c4cb-4c9d-838a-9cdab4b70800 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:25.601496 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb22008b-a931-4c79-8e37-f17a40a57708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:25.602251 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22008b-a931-4c79-8e37-f17a40a57708 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:25.602629 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22008b-a931-4c79-8e37-f17a40a57708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:25.602975 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22008b-a931-4c79-8e37-f17a40a57708 admin admin] PATH_INFO: `/v3/projects/f7d4f2d734d0480c80f4362c0b69cfb4/users/d006b09dfd6f4a38a7841918f9f8da57/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:25.621437 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb22008b-a931-4c79-8e37-f17a40a57708 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7d4f2d734d0480c80f4362c0b69cfb4, user_id=d006b09dfd6f4a38a7841918f9f8da57, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:25.626067 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb22008b-a931-4c79-8e37-f17a40a57708 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:25.712714 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1423/2863] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 13:01:25 2022] PUT /identity/v3/projects/f7d4f2d734d0480c80f4362c0b69cfb4/users/d006b09dfd6f4a38a7841918f9f8da57/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:01:25.714384 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b4d6b-fb5e-466d-8cf2-86931efcf5ae None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:25.714692 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b4d6b-fb5e-466d-8cf2-86931efcf5ae None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:25.715004 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b4d6b-fb5e-466d-8cf2-86931efcf5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:25.751758 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-ed1b4d6b-fb5e-466d-8cf2-86931efcf5ae None None] MFA Rules not processed for user `d006b09dfd6f4a38a7841918f9f8da57`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:01:25.753119 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-ed1b4d6b-fb5e-466d-8cf2-86931efcf5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:25.789320 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:01:25.855370 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1441/2864] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 13:01:25 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:01:25.857743 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb22008b-a931-4c79-8e37-f17a40a57708 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:25.927928 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb22008b-a931-4c79-8e37-f17a40a57708 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:25.932631 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334c100-799e-4076-b0b0-1c269e4285c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:25.933379 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334c100-799e-4076-b0b0-1c269e4285c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:25.933664 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334c100-799e-4076-b0b0-1c269e4285c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:25.935172 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334c100-799e-4076-b0b0-1c269e4285c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:25.935172 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6334c100-799e-4076-b0b0-1c269e4285c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:25.936548 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6334c100-799e-4076-b0b0-1c269e4285c6 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:26.068137 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1424/2865] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 13:01:25 2022] POST /identity/v3/projects => generated 403 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:01:26.069392 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b4d6b-fb5e-466d-8cf2-86931efcf5ae None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:26.080884 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b4d6b-fb5e-466d-8cf2-86931efcf5ae None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:26.142118 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67722c7-10f0-421c-978e-1bba717f9a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:26.142733 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67722c7-10f0-421c-978e-1bba717f9a6c admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:26.143012 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67722c7-10f0-421c-978e-1bba717f9a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:26.143282 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67722c7-10f0-421c-978e-1bba717f9a6c admin admin] PATH_INFO: `/v3/users` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:26.143954 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67722c7-10f0-421c-978e-1bba717f9a6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:26.146150 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67722c7-10f0-421c-978e-1bba717f9a6c admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:26.252505 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1442/2866] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 13:01:26 2022] POST /identity/v3/users => generated 308 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:01:26.256812 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334c100-799e-4076-b0b0-1c269e4285c6 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:26.267185 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334c100-799e-4076-b0b0-1c269e4285c6 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:26.272038 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5d3bbd-63c8-4d87-b505-c180695ca783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:26.272639 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d3bbd-63c8-4d87-b505-c180695ca783 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:26.272921 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d3bbd-63c8-4d87-b505-c180695ca783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:26.273200 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d3bbd-63c8-4d87-b505-c180695ca783 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:26.273994 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5d3bbd-63c8-4d87-b505-c180695ca783 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:26.275456 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5d3bbd-63c8-4d87-b505-c180695ca783 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:26.290369 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1425/2867] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 13:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:01:26.293624 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67722c7-10f0-421c-978e-1bba717f9a6c admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:26.294143 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-d67722c7-10f0-421c-978e-1bba717f9a6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:26.294731 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5d3bbd-63c8-4d87-b505-c180695ca783 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:26.310154 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67722c7-10f0-421c-978e-1bba717f9a6c admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:26.312301 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5d3bbd-63c8-4d87-b505-c180695ca783 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:26.315012 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc268a-78c6-4433-9aaa-7cb4ea3119ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:26.315728 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc268a-78c6-4433-9aaa-7cb4ea3119ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:26.315941 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b315af-02d5-4024-bf1e-08b67eabe29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:26.316134 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc268a-78c6-4433-9aaa-7cb4ea3119ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:26.316324 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc268a-78c6-4433-9aaa-7cb4ea3119ff admin admin] PATH_INFO: `/v3/users/9a4a9a635aaf44c7b81224a6b0ad9dec` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:26.316631 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b315af-02d5-4024-bf1e-08b67eabe29b admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:26.316793 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b315af-02d5-4024-bf1e-08b67eabe29b admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:26.317105 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b315af-02d5-4024-bf1e-08b67eabe29b admin admin] PATH_INFO: `/v3/projects/c5dec036a2404304bdbf0a435cf1a20c/users/902cba28a103426cbe3cba2edb48a45a/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:26.317781 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc268a-78c6-4433-9aaa-7cb4ea3119ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a4a9a635aaf44c7b81224a6b0ad9dec)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:26.319899 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc268a-78c6-4433-9aaa-7cb4ea3119ff admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:26.333381 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b315af-02d5-4024-bf1e-08b67eabe29b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5dec036a2404304bdbf0a435cf1a20c, user_id=902cba28a103426cbe3cba2edb48a45a, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:26.336212 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b315af-02d5-4024-bf1e-08b67eabe29b admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:26.416523 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1426/2868] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 13:01:26 2022] PUT /identity/v3/projects/c5dec036a2404304bdbf0a435cf1a20c/users/902cba28a103426cbe3cba2edb48a45a/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:01:26.420520 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b315af-02d5-4024-bf1e-08b67eabe29b admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:26.462760 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:01:26.481666 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b315af-02d5-4024-bf1e-08b67eabe29b admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:26.486574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d61756-cc67-4e3e-9f8b-2aeabf0e7980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:26.487215 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d61756-cc67-4e3e-9f8b-2aeabf0e7980 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:26.487523 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d61756-cc67-4e3e-9f8b-2aeabf0e7980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:26.487843 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d61756-cc67-4e3e-9f8b-2aeabf0e7980 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:26.488524 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d61756-cc67-4e3e-9f8b-2aeabf0e7980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:26.490211 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d61756-cc67-4e3e-9f8b-2aeabf0e7980 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:26.507499 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1427/2869] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 13:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:01:26.511499 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d61756-cc67-4e3e-9f8b-2aeabf0e7980 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:26.521434 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d61756-cc67-4e3e-9f8b-2aeabf0e7980 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:26.526471 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a9ff4a-7f12-4aeb-99f2-a4abf1c34520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:26.527103 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9ff4a-7f12-4aeb-99f2-a4abf1c34520 admin admin] REQUEST_METHOD: `PUT` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:26.527408 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9ff4a-7f12-4aeb-99f2-a4abf1c34520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:26.527683 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9ff4a-7f12-4aeb-99f2-a4abf1c34520 admin admin] PATH_INFO: `/v3/projects/c5dec036a2404304bdbf0a435cf1a20c/users/902cba28a103426cbe3cba2edb48a45a/roles/7933c45dcdd54ef0a98018ba8a8bddaa` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:26.529570 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a9ff4a-7f12-4aeb-99f2-a4abf1c34520 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5dec036a2404304bdbf0a435cf1a20c, user_id=902cba28a103426cbe3cba2edb48a45a, role_id=7933c45dcdd54ef0a98018ba8a8bddaa)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:26.532723 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a9ff4a-7f12-4aeb-99f2-a4abf1c34520 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:26.541321 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-63dc268a-78c6-4433-9aaa-7cb4ea3119ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a4a9a635aaf44c7b81224a6b0ad9dec'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:26.543814 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1428/2870] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 13:01:26 2022] PUT /identity/v3/projects/c5dec036a2404304bdbf0a435cf1a20c/users/902cba28a103426cbe3cba2edb48a45a/roles/7933c45dcdd54ef0a98018ba8a8bddaa => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:01:26.549339 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-63dc268a-78c6-4433-9aaa-7cb4ea3119ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a4a9a635aaf44c7b81224a6b0ad9dec'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:26.554389 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7155e0a-e25e-4f75-b2fb-0d05fc03210f None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:26.554680 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7155e0a-e25e-4f75-b2fb-0d05fc03210f None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:26.554961 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7155e0a-e25e-4f75-b2fb-0d05fc03210f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:26.561763 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-63dc268a-78c6-4433-9aaa-7cb4ea3119ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a4a9a635aaf44c7b81224a6b0ad9dec'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:26.562475 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-63dc268a-78c6-4433-9aaa-7cb4ea3119ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a4a9a635aaf44c7b81224a6b0ad9dec'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:26.585057 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-d7155e0a-e25e-4f75-b2fb-0d05fc03210f None None] MFA Rules not processed for user `902cba28a103426cbe3cba2edb48a45a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:01:26.586282 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d7155e0a-e25e-4f75-b2fb-0d05fc03210f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:26.632258 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-63dc268a-78c6-4433-9aaa-7cb4ea3119ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a4a9a635aaf44c7b81224a6b0ad9dec'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:26.642189 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1443/2871] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 13:01:26 2022] DELETE /identity/v3/users/9a4a9a635aaf44c7b81224a6b0ad9dec => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:01:26.655759 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1429/2872] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 13:01:26 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:01:26.666554 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc268a-78c6-4433-9aaa-7cb4ea3119ff admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:26.667968 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-63dc268a-78c6-4433-9aaa-7cb4ea3119ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:26.699694 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:01:26.726394 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc268a-78c6-4433-9aaa-7cb4ea3119ff admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:26.730442 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3121c3-116b-44fe-8284-9356a2d75f08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:26.731055 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3121c3-116b-44fe-8284-9356a2d75f08 service cinder] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:26.731367 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3121c3-116b-44fe-8284-9356a2d75f08 service cinder] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:26.731648 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3121c3-116b-44fe-8284-9356a2d75f08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:26.794429 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3121c3-116b-44fe-8284-9356a2d75f08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:26.795945 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3121c3-116b-44fe-8284-9356a2d75f08 service cinder] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:26.807229 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1444/2873] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 13:01:26 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:01:26.958175 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7155e0a-e25e-4f75-b2fb-0d05fc03210f None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:26.959691 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d7155e0a-e25e-4f75-b2fb-0d05fc03210f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:27.030490 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7155e0a-e25e-4f75-b2fb-0d05fc03210f None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:27.080092 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:01:27.086798 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e8992-05f9-43d5-9cce-82b44052f1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:27.087471 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e8992-05f9-43d5-9cce-82b44052f1f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:27.087751 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e8992-05f9-43d5-9cce-82b44052f1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:27.088038 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e8992-05f9-43d5-9cce-82b44052f1f8 admin admin] PATH_INFO: `/v3/users/d006b09dfd6f4a38a7841918f9f8da57` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:27.089182 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738e8992-05f9-43d5-9cce-82b44052f1f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d006b09dfd6f4a38a7841918f9f8da57)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:27.093489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738e8992-05f9-43d5-9cce-82b44052f1f8 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:27.120953 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3121c3-116b-44fe-8284-9356a2d75f08 service cinder] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:27.122449 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-4e3121c3-116b-44fe-8284-9356a2d75f08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:27.137906 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3121c3-116b-44fe-8284-9356a2d75f08 service cinder] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:27.143348 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41f306f-deec-4823-98cb-032ba8c84853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:27.144098 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41f306f-deec-4823-98cb-032ba8c84853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:27.144309 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41f306f-deec-4823-98cb-032ba8c84853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:27.144580 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41f306f-deec-4823-98cb-032ba8c84853 admin admin] PATH_INFO: `/v3/projects/a0a4271779a04986bbd34e2e989f3603` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:27.146796 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41f306f-deec-4823-98cb-032ba8c84853 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0a4271779a04986bbd34e2e989f3603)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:27.148745 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41f306f-deec-4823-98cb-032ba8c84853 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:27.272170 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-c41f306f-deec-4823-98cb-032ba8c84853 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0a4271779a04986bbd34e2e989f3603', 'request_id': 'req-c41f306f-deec-4823-98cb-032ba8c84853'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:27.274378 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-738e8992-05f9-43d5-9cce-82b44052f1f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd006b09dfd6f4a38a7841918f9f8da57'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:27.278881 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-c41f306f-deec-4823-98cb-032ba8c84853 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0a4271779a04986bbd34e2e989f3603', 'request_id': 'req-c41f306f-deec-4823-98cb-032ba8c84853'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:27.288605 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-738e8992-05f9-43d5-9cce-82b44052f1f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd006b09dfd6f4a38a7841918f9f8da57'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:27.289250 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-738e8992-05f9-43d5-9cce-82b44052f1f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd006b09dfd6f4a38a7841918f9f8da57'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:27.296208 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-738e8992-05f9-43d5-9cce-82b44052f1f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd006b09dfd6f4a38a7841918f9f8da57'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:27.370071 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-c41f306f-deec-4823-98cb-032ba8c84853 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0a4271779a04986bbd34e2e989f3603', 'request_id': 'req-c41f306f-deec-4823-98cb-032ba8c84853'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:27.381737 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-c41f306f-deec-4823-98cb-032ba8c84853 admin admin] The token cache is being invalidate because project a0a4271779a04986bbd34e2e989f3603 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:01:27.382032 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-c41f306f-deec-4823-98cb-032ba8c84853 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:27.382690 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-c41f306f-deec-4823-98cb-032ba8c84853 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:27.384420 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1445/2874] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 13:01:27 2022] DELETE /identity/v3/projects/a0a4271779a04986bbd34e2e989f3603 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:01:27.388000 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41f306f-deec-4823-98cb-032ba8c84853 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:27.389319 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-c41f306f-deec-4823-98cb-032ba8c84853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:27.426812 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-738e8992-05f9-43d5-9cce-82b44052f1f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd006b09dfd6f4a38a7841918f9f8da57'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:27.433254 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1430/2875] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 13:01:26 2022] DELETE /identity/v3/users/d006b09dfd6f4a38a7841918f9f8da57 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:01:27.437521 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa6aec-e9d4-400b-bc02-559265961043 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:27.437979 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa6aec-e9d4-400b-bc02-559265961043 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:27.438342 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa6aec-e9d4-400b-bc02-559265961043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:27.457193 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-f7aa6aec-e9d4-400b-bc02-559265961043 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:01:27.459073 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-f7aa6aec-e9d4-400b-bc02-559265961043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:27.459387 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41f306f-deec-4823-98cb-032ba8c84853 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:27.463493 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:01:27.517943 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9909b520-31eb-4b62-b857-29cafc544568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:27.518586 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9909b520-31eb-4b62-b857-29cafc544568 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:27.518874 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9909b520-31eb-4b62-b857-29cafc544568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:27.519148 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9909b520-31eb-4b62-b857-29cafc544568 admin admin] PATH_INFO: `/v3/users/5502fe40e74f461aa1bbc6f563dc5654` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:27.521041 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9909b520-31eb-4b62-b857-29cafc544568 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5502fe40e74f461aa1bbc6f563dc5654)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:27.522974 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9909b520-31eb-4b62-b857-29cafc544568 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:27.523428 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1431/2876] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 13:01:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:01:27.531152 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa6aec-e9d4-400b-bc02-559265961043 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:27.532576 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-f7aa6aec-e9d4-400b-bc02-559265961043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:27.590287 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa6aec-e9d4-400b-bc02-559265961043 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:27.593974 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a9dbb-3da3-4ade-a60c-0464fc512297 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:27.594586 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a9dbb-3da3-4ade-a60c-0464fc512297 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:27.594865 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a9dbb-3da3-4ade-a60c-0464fc512297 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:27.595138 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a9dbb-3da3-4ade-a60c-0464fc512297 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:27.604858 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a9dbb-3da3-4ade-a60c-0464fc512297 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:27.606390 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a9dbb-3da3-4ade-a60c-0464fc512297 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:27.616574 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1432/2877] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 13:01:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:01:27.692816 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-9909b520-31eb-4b62-b857-29cafc544568 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5502fe40e74f461aa1bbc6f563dc5654'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:27.699969 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-9909b520-31eb-4b62-b857-29cafc544568 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5502fe40e74f461aa1bbc6f563dc5654'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:27.713016 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-9909b520-31eb-4b62-b857-29cafc544568 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5502fe40e74f461aa1bbc6f563dc5654'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:27.713749 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-9909b520-31eb-4b62-b857-29cafc544568 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5502fe40e74f461aa1bbc6f563dc5654'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:27.788796 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-9909b520-31eb-4b62-b857-29cafc544568 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5502fe40e74f461aa1bbc6f563dc5654'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:27.796135 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1446/2878] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 13:01:27 2022] DELETE /identity/v3/users/5502fe40e74f461aa1bbc6f563dc5654 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:01:28.080456 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a9dbb-3da3-4ade-a60c-0464fc512297 service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:28.083777 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-c44a9dbb-3da3-4ade-a60c-0464fc512297 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:28.086202 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9909b520-31eb-4b62-b857-29cafc544568 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:28.088005 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-9909b520-31eb-4b62-b857-29cafc544568 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:28.099435 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a9dbb-3da3-4ade-a60c-0464fc512297 service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:28.111733 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:01:28.162268 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c925c-9be7-46c9-8482-68a9b77052bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:28.168489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c925c-9be7-46c9-8482-68a9b77052bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:28.168489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c925c-9be7-46c9-8482-68a9b77052bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:28.168489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c925c-9be7-46c9-8482-68a9b77052bb admin admin] PATH_INFO: `/v3/projects/f7d4f2d734d0480c80f4362c0b69cfb4` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:28.168489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8c925c-9be7-46c9-8482-68a9b77052bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7d4f2d734d0480c80f4362c0b69cfb4)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:28.168489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8c925c-9be7-46c9-8482-68a9b77052bb admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:28.177563 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9909b520-31eb-4b62-b857-29cafc544568 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:28.182484 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59a54e-765a-40a9-9e47-f2caf4a68b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:28.183123 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59a54e-765a-40a9-9e47-f2caf4a68b15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:28.183435 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59a54e-765a-40a9-9e47-f2caf4a68b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:28.183715 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59a54e-765a-40a9-9e47-f2caf4a68b15 admin admin] PATH_INFO: `/v3/projects/a7879e312b194158b24bdba87ab811c9` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:28.184831 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b59a54e-765a-40a9-9e47-f2caf4a68b15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7879e312b194158b24bdba87ab811c9)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:28.186774 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b59a54e-765a-40a9-9e47-f2caf4a68b15 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:28.275625 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-bd8c925c-9be7-46c9-8482-68a9b77052bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7d4f2d734d0480c80f4362c0b69cfb4', 'request_id': 'req-bd8c925c-9be7-46c9-8482-68a9b77052bb'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:28.281143 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-bd8c925c-9be7-46c9-8482-68a9b77052bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7d4f2d734d0480c80f4362c0b69cfb4', 'request_id': 'req-bd8c925c-9be7-46c9-8482-68a9b77052bb'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:28.288396 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:01:28.303701 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-4b59a54e-765a-40a9-9e47-f2caf4a68b15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7879e312b194158b24bdba87ab811c9', 'request_id': 'req-4b59a54e-765a-40a9-9e47-f2caf4a68b15'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:28.310037 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-4b59a54e-765a-40a9-9e47-f2caf4a68b15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7879e312b194158b24bdba87ab811c9', 'request_id': 'req-4b59a54e-765a-40a9-9e47-f2caf4a68b15'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:28.354374 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-bd8c925c-9be7-46c9-8482-68a9b77052bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7d4f2d734d0480c80f4362c0b69cfb4', 'request_id': 'req-bd8c925c-9be7-46c9-8482-68a9b77052bb'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:28.366694 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-bd8c925c-9be7-46c9-8482-68a9b77052bb admin admin] The token cache is being invalidate because project f7d4f2d734d0480c80f4362c0b69cfb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81805) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:01:28.366981 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-bd8c925c-9be7-46c9-8482-68a9b77052bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:28.367649 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-bd8c925c-9be7-46c9-8482-68a9b77052bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:28.369477 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1433/2879] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 13:01:28 2022] DELETE /identity/v3/projects/f7d4f2d734d0480c80f4362c0b69cfb4 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:01:28.373337 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c925c-9be7-46c9-8482-68a9b77052bb admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:28.374886 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-bd8c925c-9be7-46c9-8482-68a9b77052bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:28.439500 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c925c-9be7-46c9-8482-68a9b77052bb admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:28.463905 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-4b59a54e-765a-40a9-9e47-f2caf4a68b15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7879e312b194158b24bdba87ab811c9', 'request_id': 'req-4b59a54e-765a-40a9-9e47-f2caf4a68b15'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:28.476668 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-4b59a54e-765a-40a9-9e47-f2caf4a68b15 admin admin] The token cache is being invalidate because project a7879e312b194158b24bdba87ab811c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:01:28.476964 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-4b59a54e-765a-40a9-9e47-f2caf4a68b15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:28.477648 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-4b59a54e-765a-40a9-9e47-f2caf4a68b15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:28.479153 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1447/2880] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 13:01:28 2022] DELETE /identity/v3/projects/a7879e312b194158b24bdba87ab811c9 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:01:28.493423 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:01:28.501179 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74c6ce4-9fc2-4a8b-b742-c86d3fc9da7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:28.501857 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c6ce4-9fc2-4a8b-b742-c86d3fc9da7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:28.502137 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c6ce4-9fc2-4a8b-b742-c86d3fc9da7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:28.502412 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c6ce4-9fc2-4a8b-b742-c86d3fc9da7d admin admin] PATH_INFO: `/v3/users/902cba28a103426cbe3cba2edb48a45a` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:28.503693 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74c6ce4-9fc2-4a8b-b742-c86d3fc9da7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=902cba28a103426cbe3cba2edb48a45a)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:28.506604 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74c6ce4-9fc2-4a8b-b742-c86d3fc9da7d admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:28.691298 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c74c6ce4-9fc2-4a8b-b742-c86d3fc9da7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '902cba28a103426cbe3cba2edb48a45a'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:28.705683 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c74c6ce4-9fc2-4a8b-b742-c86d3fc9da7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '902cba28a103426cbe3cba2edb48a45a'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:28.706449 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c74c6ce4-9fc2-4a8b-b742-c86d3fc9da7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '902cba28a103426cbe3cba2edb48a45a'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:28.713976 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c74c6ce4-9fc2-4a8b-b742-c86d3fc9da7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '902cba28a103426cbe3cba2edb48a45a'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:28.792512 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-c74c6ce4-9fc2-4a8b-b742-c86d3fc9da7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '902cba28a103426cbe3cba2edb48a45a'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:28.799046 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1434/2881] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 13:01:28 2022] DELETE /identity/v3/users/902cba28a103426cbe3cba2edb48a45a => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:01:29.212984 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59a54e-765a-40a9-9e47-f2caf4a68b15 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:29.215022 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-4b59a54e-765a-40a9-9e47-f2caf4a68b15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:29.230539 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59a54e-765a-40a9-9e47-f2caf4a68b15 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:29.298778 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564e9928-3191-40d7-ab39-cdd9570535ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:29.299528 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e9928-3191-40d7-ab39-cdd9570535ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:29.299814 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e9928-3191-40d7-ab39-cdd9570535ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:29.300180 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e9928-3191-40d7-ab39-cdd9570535ee admin admin] PATH_INFO: `/v3/projects/c5dec036a2404304bdbf0a435cf1a20c` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:29.301294 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564e9928-3191-40d7-ab39-cdd9570535ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5dec036a2404304bdbf0a435cf1a20c)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:29.304925 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564e9928-3191-40d7-ab39-cdd9570535ee admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:29.424207 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-564e9928-3191-40d7-ab39-cdd9570535ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5dec036a2404304bdbf0a435cf1a20c', 'request_id': 'req-564e9928-3191-40d7-ab39-cdd9570535ee'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:29.430665 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-564e9928-3191-40d7-ab39-cdd9570535ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5dec036a2404304bdbf0a435cf1a20c', 'request_id': 'req-564e9928-3191-40d7-ab39-cdd9570535ee'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:29.491538 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-564e9928-3191-40d7-ab39-cdd9570535ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5dec036a2404304bdbf0a435cf1a20c', 'request_id': 'req-564e9928-3191-40d7-ab39-cdd9570535ee'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:29.502672 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-564e9928-3191-40d7-ab39-cdd9570535ee admin admin] The token cache is being invalidate because project c5dec036a2404304bdbf0a435cf1a20c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:01:29.503098 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-564e9928-3191-40d7-ab39-cdd9570535ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:29.503780 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-564e9928-3191-40d7-ab39-cdd9570535ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:29.505463 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1448/2882] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 13:01:29 2022] DELETE /identity/v3/projects/c5dec036a2404304bdbf0a435cf1a20c => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:01:29.510396 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4af4df-f0d8-468e-8ced-43d0aae78b65 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:29.510881 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4af4df-f0d8-468e-8ced-43d0aae78b65 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:29.511012 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4af4df-f0d8-468e-8ced-43d0aae78b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:29.531336 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-2f4af4df-f0d8-468e-8ced-43d0aae78b65 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:01:29.532542 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-2f4af4df-f0d8-468e-8ced-43d0aae78b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:29.591868 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1435/2883] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 13:01:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:01:29.600544 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdd5ac-875d-4f53-ad5a-b2c70b83ebe0 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:29.600917 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdd5ac-875d-4f53-ad5a-b2c70b83ebe0 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:29.601205 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdd5ac-875d-4f53-ad5a-b2c70b83ebe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:29.621591 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-6fcdd5ac-875d-4f53-ad5a-b2c70b83ebe0 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:01:29.622799 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-6fcdd5ac-875d-4f53-ad5a-b2c70b83ebe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:29.631117 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1449/2884] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 13:01:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:01:29.635336 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4af4df-f0d8-468e-8ced-43d0aae78b65 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:29.644902 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4af4df-f0d8-468e-8ced-43d0aae78b65 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:29.650580 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dddfa1f-d8a5-469a-99ae-8a06fe469eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:29.651189 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dddfa1f-d8a5-469a-99ae-8a06fe469eb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:29.651510 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dddfa1f-d8a5-469a-99ae-8a06fe469eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:29.651787 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dddfa1f-d8a5-469a-99ae-8a06fe469eb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:29.652436 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dddfa1f-d8a5-469a-99ae-8a06fe469eb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:01:29.652739 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dddfa1f-d8a5-469a-99ae-8a06fe469eb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:29.654626 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dddfa1f-d8a5-469a-99ae-8a06fe469eb9 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:29.670510 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1436/2885] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 13:01:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:01:29.674941 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcdd5ac-875d-4f53-ad5a-b2c70b83ebe0 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:29.684403 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcdd5ac-875d-4f53-ad5a-b2c70b83ebe0 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:29.689071 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1532f69-7abc-46ce-822f-6217f28979c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:29.689668 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1532f69-7abc-46ce-822f-6217f28979c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:29.689948 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1532f69-7abc-46ce-822f-6217f28979c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:29.690223 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1532f69-7abc-46ce-822f-6217f28979c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:29.690848 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1532f69-7abc-46ce-822f-6217f28979c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:29.692789 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1532f69-7abc-46ce-822f-6217f28979c0 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:29.778659 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1450/2886] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 13:01:29 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:01:29.782959 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dddfa1f-d8a5-469a-99ae-8a06fe469eb9 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:29.793006 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dddfa1f-d8a5-469a-99ae-8a06fe469eb9 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:29.848357 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838dd37f-cd37-4c0d-8abe-2cd7c28817f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:29.849036 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838dd37f-cd37-4c0d-8abe-2cd7c28817f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:29.849342 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838dd37f-cd37-4c0d-8abe-2cd7c28817f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:29.849615 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838dd37f-cd37-4c0d-8abe-2cd7c28817f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:29.850232 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838dd37f-cd37-4c0d-8abe-2cd7c28817f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:29.852160 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838dd37f-cd37-4c0d-8abe-2cd7c28817f4 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:29.860968 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:01:29.972168 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1437/2887] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 13:01:29 2022] POST /identity/v3/users => generated 288 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:01:29.976423 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1532f69-7abc-46ce-822f-6217f28979c0 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:29.986726 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1532f69-7abc-46ce-822f-6217f28979c0 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:29.991308 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81faf6e-f357-487d-97f0-598853df5152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:29.991936 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81faf6e-f357-487d-97f0-598853df5152 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:29.992217 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81faf6e-f357-487d-97f0-598853df5152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:29.992506 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81faf6e-f357-487d-97f0-598853df5152 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:29.993189 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81faf6e-f357-487d-97f0-598853df5152 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:29.995203 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81faf6e-f357-487d-97f0-598853df5152 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:30.005636 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1451/2888] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 13:01:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:01:30.009540 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838dd37f-cd37-4c0d-8abe-2cd7c28817f4 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:30.019553 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838dd37f-cd37-4c0d-8abe-2cd7c28817f4 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:30.025181 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149ad6c3-f7e0-47ef-8f19-b6d3b6815283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:30.025778 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ad6c3-f7e0-47ef-8f19-b6d3b6815283 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:30.026055 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ad6c3-f7e0-47ef-8f19-b6d3b6815283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:30.026327 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ad6c3-f7e0-47ef-8f19-b6d3b6815283 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:30.026958 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149ad6c3-f7e0-47ef-8f19-b6d3b6815283 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:30.028599 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149ad6c3-f7e0-47ef-8f19-b6d3b6815283 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:30.040313 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1438/2889] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 13:01:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:01:30.043756 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81faf6e-f357-487d-97f0-598853df5152 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:30.054656 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81faf6e-f357-487d-97f0-598853df5152 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:30.059561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5b977-1964-40d3-af8b-5756b3c02c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:30.060213 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5b977-1964-40d3-af8b-5756b3c02c05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:30.060498 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5b977-1964-40d3-af8b-5756b3c02c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:30.060775 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5b977-1964-40d3-af8b-5756b3c02c05 admin admin] PATH_INFO: `/v3/projects/a78ec63a97844229a899d5e55a432c8a/users/6ccc3af44b3c4488b64d7453f5d48aa9/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:30.063074 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:01:30.076330 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a5b977-1964-40d3-af8b-5756b3c02c05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a78ec63a97844229a899d5e55a432c8a, user_id=6ccc3af44b3c4488b64d7453f5d48aa9, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:30.078973 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a5b977-1964-40d3-af8b-5756b3c02c05 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:30.148210 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1452/2890] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 13:01:30 2022] PUT /identity/v3/projects/a78ec63a97844229a899d5e55a432c8a/users/6ccc3af44b3c4488b64d7453f5d48aa9/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:01:30.159744 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b311c-c099-487e-8cd4-23010e9ea095 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:30.160025 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b311c-c099-487e-8cd4-23010e9ea095 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:30.160297 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b311c-c099-487e-8cd4-23010e9ea095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:30.189354 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-440b311c-c099-487e-8cd4-23010e9ea095 None None] MFA Rules not processed for user `6ccc3af44b3c4488b64d7453f5d48aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:01:30.190564 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-440b311c-c099-487e-8cd4-23010e9ea095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:30.257986 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1439/2891] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 13:01:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:01:30.269905 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5b977-1964-40d3-af8b-5756b3c02c05 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:30.272167 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-49a5b977-1964-40d3-af8b-5756b3c02c05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:30.356733 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5b977-1964-40d3-af8b-5756b3c02c05 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:30.360425 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce589df-0b49-4e11-bead-0f5690e4ce9b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b697e4847b13496d98a543675f5df1c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:30.361032 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce589df-0b49-4e11-bead-0f5690e4ce9b service glance] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:30.361310 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce589df-0b49-4e11-bead-0f5690e4ce9b service glance] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:30.361580 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce589df-0b49-4e11-bead-0f5690e4ce9b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:30.370751 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce589df-0b49-4e11-bead-0f5690e4ce9b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:30.372194 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce589df-0b49-4e11-bead-0f5690e4ce9b service glance] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:30.382378 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1453/2892] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 13:01:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:01:40.253026 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:INSERT=2,keystone:SELECT=39 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:01:40.376813 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=42,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:01:47.556801 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440b311c-c099-487e-8cd4-23010e9ea095 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:47.620686 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440b311c-c099-487e-8cd4-23010e9ea095 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:47.625248 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cabf91-b034-41f1-9738-578da89026bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:47.625882 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cabf91-b034-41f1-9738-578da89026bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:47.626163 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cabf91-b034-41f1-9738-578da89026bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:47.626434 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cabf91-b034-41f1-9738-578da89026bf admin admin] PATH_INFO: `/v3/users/6ccc3af44b3c4488b64d7453f5d48aa9` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:47.627619 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cabf91-b034-41f1-9738-578da89026bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ccc3af44b3c4488b64d7453f5d48aa9)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:47.629611 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cabf91-b034-41f1-9738-578da89026bf admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:47.816688 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-80cabf91-b034-41f1-9738-578da89026bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ccc3af44b3c4488b64d7453f5d48aa9'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:47.829309 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-80cabf91-b034-41f1-9738-578da89026bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ccc3af44b3c4488b64d7453f5d48aa9'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:47.829975 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-80cabf91-b034-41f1-9738-578da89026bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ccc3af44b3c4488b64d7453f5d48aa9'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:47.837750 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-80cabf91-b034-41f1-9738-578da89026bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ccc3af44b3c4488b64d7453f5d48aa9'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:47.894572 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-80cabf91-b034-41f1-9738-578da89026bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ccc3af44b3c4488b64d7453f5d48aa9'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:47.901122 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1440/2893] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 13:01:47 2022] DELETE /identity/v3/users/6ccc3af44b3c4488b64d7453f5d48aa9 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:01:47.905938 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab42f6e-7a9a-41f1-81d2-0fbcf7f511f0 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:47.906446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab42f6e-7a9a-41f1-81d2-0fbcf7f511f0 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:47.906446 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab42f6e-7a9a-41f1-81d2-0fbcf7f511f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:47.924832 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-cab42f6e-7a9a-41f1-81d2-0fbcf7f511f0 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:01:47.926027 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-cab42f6e-7a9a-41f1-81d2-0fbcf7f511f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:47.983336 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1454/2894] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 13:01:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:01:47.993558 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cabf91-b034-41f1-9738-578da89026bf admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:47.997844 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-80cabf91-b034-41f1-9738-578da89026bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:48.052378 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cabf91-b034-41f1-9738-578da89026bf admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:48.056648 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e0ad4b-9fc3-4228-b2f3-9359764bedbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:48.057252 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0ad4b-9fc3-4228-b2f3-9359764bedbc service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:48.057529 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0ad4b-9fc3-4228-b2f3-9359764bedbc service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:48.057800 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0ad4b-9fc3-4228-b2f3-9359764bedbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:48.066769 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e0ad4b-9fc3-4228-b2f3-9359764bedbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:48.068154 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e0ad4b-9fc3-4228-b2f3-9359764bedbc service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:48.078166 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1441/2895] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 13:01:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:01:48.477669 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab42f6e-7a9a-41f1-81d2-0fbcf7f511f0 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:48.478875 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-cab42f6e-7a9a-41f1-81d2-0fbcf7f511f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:48.491186 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab42f6e-7a9a-41f1-81d2-0fbcf7f511f0 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:48.497009 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5ab383-f706-4e3b-a69e-2821e6f127da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:48.497651 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ab383-f706-4e3b-a69e-2821e6f127da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:48.497927 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ab383-f706-4e3b-a69e-2821e6f127da admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:48.498196 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ab383-f706-4e3b-a69e-2821e6f127da admin admin] PATH_INFO: `/v3/projects/a78ec63a97844229a899d5e55a432c8a` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:48.499307 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5ab383-f706-4e3b-a69e-2821e6f127da admin admin] RBAC: Authorizing `identity:delete_project(project_id=a78ec63a97844229a899d5e55a432c8a)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:48.501237 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5ab383-f706-4e3b-a69e-2821e6f127da admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:48.615565 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-3e5ab383-f706-4e3b-a69e-2821e6f127da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a78ec63a97844229a899d5e55a432c8a', 'request_id': 'req-3e5ab383-f706-4e3b-a69e-2821e6f127da'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:48.622751 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-3e5ab383-f706-4e3b-a69e-2821e6f127da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a78ec63a97844229a899d5e55a432c8a', 'request_id': 'req-3e5ab383-f706-4e3b-a69e-2821e6f127da'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:48.674016 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-3e5ab383-f706-4e3b-a69e-2821e6f127da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a78ec63a97844229a899d5e55a432c8a', 'request_id': 'req-3e5ab383-f706-4e3b-a69e-2821e6f127da'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:48.684514 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-3e5ab383-f706-4e3b-a69e-2821e6f127da admin admin] The token cache is being invalidate because project a78ec63a97844229a899d5e55a432c8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:01:48.687854 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-3e5ab383-f706-4e3b-a69e-2821e6f127da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:48.687854 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-3e5ab383-f706-4e3b-a69e-2821e6f127da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:48.688100 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1455/2896] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 13:01:48 2022] DELETE /identity/v3/projects/a78ec63a97844229a899d5e55a432c8a => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:01:48.693049 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10930b6d-2904-45e9-898a-ab34e8032116 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:48.693590 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10930b6d-2904-45e9-898a-ab34e8032116 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:48.693712 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10930b6d-2904-45e9-898a-ab34e8032116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:48.714403 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-10930b6d-2904-45e9-898a-ab34e8032116 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:01:48.716561 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-10930b6d-2904-45e9-898a-ab34e8032116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:48.732397 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:01:48.774448 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1442/2897] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 13:01:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:01:48.782184 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de1558-4d6d-4a8d-a5fb-91817b92f12b None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:48.782470 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de1558-4d6d-4a8d-a5fb-91817b92f12b None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:48.782745 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de1558-4d6d-4a8d-a5fb-91817b92f12b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:48.802085 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-d7de1558-4d6d-4a8d-a5fb-91817b92f12b None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:01:48.803277 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-d7de1558-4d6d-4a8d-a5fb-91817b92f12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:48.811759 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1456/2898] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 13:01:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:01:48.816070 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10930b6d-2904-45e9-898a-ab34e8032116 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:48.826872 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10930b6d-2904-45e9-898a-ab34e8032116 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:48.831495 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45982ad4-e7d5-458a-91dc-b90535aff680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:48.832101 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45982ad4-e7d5-458a-91dc-b90535aff680 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:48.832383 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45982ad4-e7d5-458a-91dc-b90535aff680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:48.832656 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45982ad4-e7d5-458a-91dc-b90535aff680 admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:48.833293 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45982ad4-e7d5-458a-91dc-b90535aff680 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:01:48.833597 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45982ad4-e7d5-458a-91dc-b90535aff680 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:48.835185 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45982ad4-e7d5-458a-91dc-b90535aff680 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:48.847585 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1443/2899] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 13:01:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:01:48.852124 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de1558-4d6d-4a8d-a5fb-91817b92f12b None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:48.861174 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de1558-4d6d-4a8d-a5fb-91817b92f12b None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:48.865745 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c685f01-6a2a-483b-937b-e59742f031b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:48.866283 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c685f01-6a2a-483b-937b-e59742f031b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:48.866561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c685f01-6a2a-483b-937b-e59742f031b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:48.867367 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c685f01-6a2a-483b-937b-e59742f031b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:48.867995 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c685f01-6a2a-483b-937b-e59742f031b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:48.870124 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c685f01-6a2a-483b-937b-e59742f031b2 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:48.970070 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1457/2900] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 13:01:48 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:01:48.975379 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45982ad4-e7d5-458a-91dc-b90535aff680 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:48.985733 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45982ad4-e7d5-458a-91dc-b90535aff680 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:49.045175 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf12f042-de9f-42f1-b4ac-163fb95f6de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:49.045781 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12f042-de9f-42f1-b4ac-163fb95f6de2 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:49.046056 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12f042-de9f-42f1-b4ac-163fb95f6de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:49.046326 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12f042-de9f-42f1-b4ac-163fb95f6de2 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:49.047006 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf12f042-de9f-42f1-b4ac-163fb95f6de2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:49.048926 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf12f042-de9f-42f1-b4ac-163fb95f6de2 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:49.180752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1444/2901] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 13:01:48 2022] POST /identity/v3/users => generated 293 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:01:49.184650 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c685f01-6a2a-483b-937b-e59742f031b2 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:49.195112 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c685f01-6a2a-483b-937b-e59742f031b2 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:49.199978 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e186a1-acf0-4e5b-846b-77dae5693067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:49.200628 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e186a1-acf0-4e5b-846b-77dae5693067 admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:49.200916 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e186a1-acf0-4e5b-846b-77dae5693067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:49.201214 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e186a1-acf0-4e5b-846b-77dae5693067 admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:49.201888 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e186a1-acf0-4e5b-846b-77dae5693067 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:49.203561 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e186a1-acf0-4e5b-846b-77dae5693067 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:49.213475 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1458/2902] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 13:01:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:01:49.217363 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf12f042-de9f-42f1-b4ac-163fb95f6de2 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:49.227784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf12f042-de9f-42f1-b4ac-163fb95f6de2 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:49.232695 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656a959-4a3c-44f7-b79d-b67f32d91835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:49.233287 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656a959-4a3c-44f7-b79d-b67f32d91835 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:49.233563 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656a959-4a3c-44f7-b79d-b67f32d91835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:49.233833 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656a959-4a3c-44f7-b79d-b67f32d91835 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:49.234466 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f656a959-4a3c-44f7-b79d-b67f32d91835 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:49.236075 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f656a959-4a3c-44f7-b79d-b67f32d91835 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:49.248480 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1445/2903] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 13:01:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:01:49.252342 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e186a1-acf0-4e5b-846b-77dae5693067 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:49.262404 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e186a1-acf0-4e5b-846b-77dae5693067 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:49.266912 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ec026-3981-47d7-8238-5b353ab2f537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:49.267601 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ec026-3981-47d7-8238-5b353ab2f537 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:49.267886 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ec026-3981-47d7-8238-5b353ab2f537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:49.268163 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ec026-3981-47d7-8238-5b353ab2f537 admin admin] PATH_INFO: `/v3/projects/694f4462556e4c61a8162773ff0705fd/users/e633a299bd064d7f9090ca117ee32395/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:49.282595 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18ec026-3981-47d7-8238-5b353ab2f537 admin admin] RBAC: Authorizing `identity:create_grant(project_id=694f4462556e4c61a8162773ff0705fd, user_id=e633a299bd064d7f9090ca117ee32395, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:49.285339 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18ec026-3981-47d7-8238-5b353ab2f537 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:49.354279 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1459/2904] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 13:01:49 2022] PUT /identity/v3/projects/694f4462556e4c61a8162773ff0705fd/users/e633a299bd064d7f9090ca117ee32395/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:01:49.420667 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9885f-2961-4b39-ae7f-76f197a18ecb None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:49.420954 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9885f-2961-4b39-ae7f-76f197a18ecb None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:49.421239 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9885f-2961-4b39-ae7f-76f197a18ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:49.449879 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-0aa9885f-2961-4b39-ae7f-76f197a18ecb None None] MFA Rules not processed for user `e633a299bd064d7f9090ca117ee32395`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:01:49.451001 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-0aa9885f-2961-4b39-ae7f-76f197a18ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:49.513318 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1446/2905] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 13:01:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:01:49.524510 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ec026-3981-47d7-8238-5b353ab2f537 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:49.526177 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-b18ec026-3981-47d7-8238-5b353ab2f537 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:49.548334 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:01:49.582947 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ec026-3981-47d7-8238-5b353ab2f537 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:49.587239 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0192d415-f0db-475c-905d-bbfd97b26570 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:49.589471 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192d415-f0db-475c-905d-bbfd97b26570 service cinder] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:49.589471 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192d415-f0db-475c-905d-bbfd97b26570 service cinder] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:49.589471 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192d415-f0db-475c-905d-bbfd97b26570 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:49.596746 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0192d415-f0db-475c-905d-bbfd97b26570 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:49.598208 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0192d415-f0db-475c-905d-bbfd97b26570 service cinder] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:49.609089 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1460/2906] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 13:01:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:01:59.510526 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:01:59.540240 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa9885f-2961-4b39-ae7f-76f197a18ecb None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:59.603063 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa9885f-2961-4b39-ae7f-76f197a18ecb None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:59.607583 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0942784-954c-41f8-90ec-6dc1ae051d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:59.608211 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0942784-954c-41f8-90ec-6dc1ae051d14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:59.608491 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0942784-954c-41f8-90ec-6dc1ae051d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:59.608784 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0942784-954c-41f8-90ec-6dc1ae051d14 admin admin] PATH_INFO: `/v3/users/e633a299bd064d7f9090ca117ee32395` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:59.609865 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0942784-954c-41f8-90ec-6dc1ae051d14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e633a299bd064d7f9090ca117ee32395)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:01:59.611813 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0942784-954c-41f8-90ec-6dc1ae051d14 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:01:59.631650 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=17 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:01:59.745076 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e0942784-954c-41f8-90ec-6dc1ae051d14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e633a299bd064d7f9090ca117ee32395'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:59.756752 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e0942784-954c-41f8-90ec-6dc1ae051d14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e633a299bd064d7f9090ca117ee32395'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:59.757403 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e0942784-954c-41f8-90ec-6dc1ae051d14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e633a299bd064d7f9090ca117ee32395'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:59.764434 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e0942784-954c-41f8-90ec-6dc1ae051d14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e633a299bd064d7f9090ca117ee32395'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:59.828211 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-e0942784-954c-41f8-90ec-6dc1ae051d14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e633a299bd064d7f9090ca117ee32395'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:01:59.834489 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1447/2907] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 13:01:59 2022] DELETE /identity/v3/users/e633a299bd064d7f9090ca117ee32395 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:01:59.839011 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d73b7-a536-4cf4-95fb-3030909781c1 None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:59.839300 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d73b7-a536-4cf4-95fb-3030909781c1 None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:59.839598 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d73b7-a536-4cf4-95fb-3030909781c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:01:59.859438 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-992d73b7-a536-4cf4-95fb-3030909781c1 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:01:59.860637 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-992d73b7-a536-4cf4-95fb-3030909781c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:59.917870 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1461/2908] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 13:01:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:01:59.927618 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0942784-954c-41f8-90ec-6dc1ae051d14 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:01:59.929001 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-e0942784-954c-41f8-90ec-6dc1ae051d14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:01:59.985739 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0942784-954c-41f8-90ec-6dc1ae051d14 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:01:59.989366 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe7a90d-d93e-41d1-8e8b-ff2e2df1a6b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:01:59.989972 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7a90d-d93e-41d1-8e8b-ff2e2df1a6b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:01:59.990248 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7a90d-d93e-41d1-8e8b-ff2e2df1a6b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:01:59.990520 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7a90d-d93e-41d1-8e8b-ff2e2df1a6b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:02:00.000398 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe7a90d-d93e-41d1-8e8b-ff2e2df1a6b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:02:00.001823 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe7a90d-d93e-41d1-8e8b-ff2e2df1a6b3 service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:02:00.011373 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1448/2909] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 13:01:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:02:00.378917 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d73b7-a536-4cf4-95fb-3030909781c1 None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:02:00.380392 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-992d73b7-a536-4cf4-95fb-3030909781c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:02:00.393112 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d73b7-a536-4cf4-95fb-3030909781c1 None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:02:00.398279 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44153361-a501-4df0-b92f-df5c3f6b1dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:02:00.399727 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44153361-a501-4df0-b92f-df5c3f6b1dfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:02:00.400012 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44153361-a501-4df0-b92f-df5c3f6b1dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:02:00.400273 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44153361-a501-4df0-b92f-df5c3f6b1dfd admin admin] PATH_INFO: `/v3/projects/694f4462556e4c61a8162773ff0705fd` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:02:00.401628 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44153361-a501-4df0-b92f-df5c3f6b1dfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=694f4462556e4c61a8162773ff0705fd)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:02:00.403661 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44153361-a501-4df0-b92f-df5c3f6b1dfd admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:02:00.518407 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-44153361-a501-4df0-b92f-df5c3f6b1dfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '694f4462556e4c61a8162773ff0705fd', 'request_id': 'req-44153361-a501-4df0-b92f-df5c3f6b1dfd'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:02:00.525735 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-44153361-a501-4df0-b92f-df5c3f6b1dfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '694f4462556e4c61a8162773ff0705fd', 'request_id': 'req-44153361-a501-4df0-b92f-df5c3f6b1dfd'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:02:00.594350 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-44153361-a501-4df0-b92f-df5c3f6b1dfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '694f4462556e4c61a8162773ff0705fd', 'request_id': 'req-44153361-a501-4df0-b92f-df5c3f6b1dfd'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:02:00.605292 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-44153361-a501-4df0-b92f-df5c3f6b1dfd admin admin] The token cache is being invalidate because project 694f4462556e4c61a8162773ff0705fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:02:00.605580 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-44153361-a501-4df0-b92f-df5c3f6b1dfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:02:00.606192 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-44153361-a501-4df0-b92f-df5c3f6b1dfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:02:00.607687 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1462/2910] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 13:02:00 2022] DELETE /identity/v3/projects/694f4462556e4c61a8162773ff0705fd => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:02:00.613654 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795325f-0fa8-4f9e-a4c3-1ee52a9d6575 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:02:00.617458 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795325f-0fa8-4f9e-a4c3-1ee52a9d6575 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:02:00.617458 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795325f-0fa8-4f9e-a4c3-1ee52a9d6575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:02:00.634863 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-c795325f-0fa8-4f9e-a4c3-1ee52a9d6575 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:02:00.636032 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-c795325f-0fa8-4f9e-a4c3-1ee52a9d6575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:02:00.653309 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:02:00.694929 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1449/2911] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 13:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:02:00.703182 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670062b2-c6ec-4e79-8139-6a77ae00e48c None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:02:00.703700 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670062b2-c6ec-4e79-8139-6a77ae00e48c None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:02:00.703856 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670062b2-c6ec-4e79-8139-6a77ae00e48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:02:00.724867 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-670062b2-c6ec-4e79-8139-6a77ae00e48c None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:02:00.726060 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-670062b2-c6ec-4e79-8139-6a77ae00e48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:02:00.733917 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1463/2912] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 13:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:02:00.738111 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c795325f-0fa8-4f9e-a4c3-1ee52a9d6575 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:02:00.748459 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c795325f-0fa8-4f9e-a4c3-1ee52a9d6575 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:02:00.753881 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e1543-a8ab-4fba-949d-946f034c7c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:02:00.755110 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e1543-a8ab-4fba-949d-946f034c7c69 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:02:00.755734 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e1543-a8ab-4fba-949d-946f034c7c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:02:00.756327 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e1543-a8ab-4fba-949d-946f034c7c69 admin admin] PATH_INFO: `/v3/domains` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:02:00.756979 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114e1543-a8ab-4fba-949d-946f034c7c69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81805) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:02:00.757285 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114e1543-a8ab-4fba-949d-946f034c7c69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:02:00.760677 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114e1543-a8ab-4fba-949d-946f034c7c69 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:02:00.772648 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1450/2913] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 13:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:02:00.777565 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670062b2-c6ec-4e79-8139-6a77ae00e48c None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:02:00.786793 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670062b2-c6ec-4e79-8139-6a77ae00e48c None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:02:00.792402 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878ef0e-4c06-4783-8436-bc1c5524fe2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:02:00.793008 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878ef0e-4c06-4783-8436-bc1c5524fe2e admin admin] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:02:00.793316 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878ef0e-4c06-4783-8436-bc1c5524fe2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:02:00.793584 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878ef0e-4c06-4783-8436-bc1c5524fe2e admin admin] PATH_INFO: `/v3/projects` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:02:00.794223 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7878ef0e-4c06-4783-8436-bc1c5524fe2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:02:00.796135 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7878ef0e-4c06-4783-8436-bc1c5524fe2e admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:02:00.866832 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1464/2914] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 13:02:00 2022] POST /identity/v3/projects => generated 381 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:02:00.871783 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e1543-a8ab-4fba-949d-946f034c7c69 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:02:00.881712 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e1543-a8ab-4fba-949d-946f034c7c69 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:02:00.938495 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff10a8-5069-47dc-a885-c48501a81361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:02:00.939014 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff10a8-5069-47dc-a885-c48501a81361 admin admin] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:02:00.939293 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff10a8-5069-47dc-a885-c48501a81361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:02:00.939595 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff10a8-5069-47dc-a885-c48501a81361 admin admin] PATH_INFO: `/v3/users` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:02:00.940216 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff10a8-5069-47dc-a885-c48501a81361 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:02:00.942108 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff10a8-5069-47dc-a885-c48501a81361 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:02:01.066347 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1451/2915] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 13:02:00 2022] POST /identity/v3/users => generated 291 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:02:01.071927 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878ef0e-4c06-4783-8436-bc1c5524fe2e admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:02:01.082018 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878ef0e-4c06-4783-8436-bc1c5524fe2e admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:02:01.086479 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6868bd-733f-442f-8ef2-17d09e403add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:02:01.088939 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6868bd-733f-442f-8ef2-17d09e403add admin admin] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:02:01.088939 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6868bd-733f-442f-8ef2-17d09e403add admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:02:01.088939 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6868bd-733f-442f-8ef2-17d09e403add admin admin] PATH_INFO: `/v3/roles` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:02:01.088939 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6868bd-733f-442f-8ef2-17d09e403add admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:02:01.090174 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6868bd-733f-442f-8ef2-17d09e403add admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:02:01.100711 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1465/2916] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 13:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:02:01.105320 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff10a8-5069-47dc-a885-c48501a81361 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:02:01.115311 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff10a8-5069-47dc-a885-c48501a81361 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:02:01.120977 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3657ec83-4127-414f-826d-5ad31d59d5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:02:01.121567 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3657ec83-4127-414f-826d-5ad31d59d5d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:02:01.121844 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3657ec83-4127-414f-826d-5ad31d59d5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:02:01.122112 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3657ec83-4127-414f-826d-5ad31d59d5d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:02:01.122737 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3657ec83-4127-414f-826d-5ad31d59d5d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:02:01.124358 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3657ec83-4127-414f-826d-5ad31d59d5d4 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:02:01.135588 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1452/2917] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 13:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:02:01.139651 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6868bd-733f-442f-8ef2-17d09e403add admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:02:01.150253 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6868bd-733f-442f-8ef2-17d09e403add admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:02:01.154945 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d1f70-0f01-43bc-a2f5-a612f160c9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:02:01.155642 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d1f70-0f01-43bc-a2f5-a612f160c9e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:02:01.155924 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d1f70-0f01-43bc-a2f5-a612f160c9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:02:01.156198 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d1f70-0f01-43bc-a2f5-a612f160c9e6 admin admin] PATH_INFO: `/v3/projects/9421d499349f4d90b729021b00961cf2/users/5a67a58ae60a4e13a3bfe77462c4337e/roles/47d5e2ea167241d5a27c8bbf4593eead` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:02:01.171946 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9d1f70-0f01-43bc-a2f5-a612f160c9e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9421d499349f4d90b729021b00961cf2, user_id=5a67a58ae60a4e13a3bfe77462c4337e, role_id=47d5e2ea167241d5a27c8bbf4593eead)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:02:01.174606 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9d1f70-0f01-43bc-a2f5-a612f160c9e6 admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:02:01.262905 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1466/2918] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 13:02:01 2022] PUT /identity/v3/projects/9421d499349f4d90b729021b00961cf2/users/5a67a58ae60a4e13a3bfe77462c4337e/roles/47d5e2ea167241d5a27c8bbf4593eead => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:02:01.267978 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb132f-a9c8-4ebc-9692-e39877096244 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:02:01.268261 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb132f-a9c8-4ebc-9692-e39877096244 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:02:01.268558 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb132f-a9c8-4ebc-9692-e39877096244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:02:01.287230 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-a9bb132f-a9c8-4ebc-9692-e39877096244 None None] MFA Rules not processed for user `77a14102256040ab81672f6adeaa6c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:02:01.288448 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-a9bb132f-a9c8-4ebc-9692-e39877096244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:02:01.347366 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1453/2919] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 13:02:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:02:01.357434 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d1f70-0f01-43bc-a2f5-a612f160c9e6 admin admin] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:02:01.359112 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-2d9d1f70-0f01-43bc-a2f5-a612f160c9e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:02:01.381352 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:02:01.418430 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d1f70-0f01-43bc-a2f5-a612f160c9e6 admin admin] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:02:01.422423 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c8fec1-a879-4166-b9ab-f1524b024611 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:02:01.426822 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c8fec1-a879-4166-b9ab-f1524b024611 service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:02:01.426822 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c8fec1-a879-4166-b9ab-f1524b024611 service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:02:01.426822 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c8fec1-a879-4166-b9ab-f1524b024611 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:02:01.433486 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c8fec1-a879-4166-b9ab-f1524b024611 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:02:01.434904 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c8fec1-a879-4166-b9ab-f1524b024611 service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:02:01.444586 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1467/2920] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 13:02:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:02:02.675463 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd7a92-172c-4b6f-ade4-5b7c651b4302 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:02:02.675959 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd7a92-172c-4b6f-ade4-5b7c651b4302 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:02:02.676065 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd7a92-172c-4b6f-ade4-5b7c651b4302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:02:02.707188 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-60bd7a92-172c-4b6f-ade4-5b7c651b4302 None None] MFA Rules not processed for user `5a67a58ae60a4e13a3bfe77462c4337e`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:02:02.708436 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-60bd7a92-172c-4b6f-ade4-5b7c651b4302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:02:02.725757 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:02:02.775174 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1454/2921] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 13:02:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:02:02.786477 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c8fec1-a879-4166-b9ab-f1524b024611 service neutron] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:02:02.787929 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-93c8fec1-a879-4166-b9ab-f1524b024611 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:02:02.845334 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c8fec1-a879-4166-b9ab-f1524b024611 service neutron] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:02:02.849043 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ceb525-d573-4c7c-802e-8b2d12fbefbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524e9184eee1486cb071cad5c67050c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:02:02.849653 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ceb525-d573-4c7c-802e-8b2d12fbefbe service cinder] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:02:02.849936 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ceb525-d573-4c7c-802e-8b2d12fbefbe service cinder] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:02:02.850211 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ceb525-d573-4c7c-802e-8b2d12fbefbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:02:02.859137 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ceb525-d573-4c7c-802e-8b2d12fbefbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:02:02.860629 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ceb525-d573-4c7c-802e-8b2d12fbefbe service cinder] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:02:02.870106 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1468/2922] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 13:02:02 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:02:04.547683 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f7982-97e9-4342-92c4-1603acbf7b47 None None] REQUEST_METHOD: `POST` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:02:04.548242 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f7982-97e9-4342-92c4-1603acbf7b47 None None] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:02:04.548242 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f7982-97e9-4342-92c4-1603acbf7b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:02:04.566830 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.auth.core [None req-ce5f7982-97e9-4342-92c4-1603acbf7b47 None None] MFA Rules not processed for user `5a67a58ae60a4e13a3bfe77462c4337e`. Rule list: `[]` (Enabled: `True`). {{(pid=81805) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:02:04.568048 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ce5f7982-97e9-4342-92c4-1603acbf7b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:02:04.575021 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1455/2923] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 13:02:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:02:04.813050 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ceb525-d573-4c7c-802e-8b2d12fbefbe service cinder] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:02:04.822554 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ceb525-d573-4c7c-802e-8b2d12fbefbe service cinder] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:02:04.826159 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e652fc-7966-4e28-9965-1de1fe4af490 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:02:04.826750 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e652fc-7966-4e28-9965-1de1fe4af490 service neutron] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:02:04.827031 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e652fc-7966-4e28-9965-1de1fe4af490 service neutron] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:02:04.827304 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e652fc-7966-4e28-9965-1de1fe4af490 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:02:04.829019 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-a1e652fc-7966-4e28-9965-1de1fe4af490 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:02:04.839009 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e652fc-7966-4e28-9965-1de1fe4af490 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:02:04.840528 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e652fc-7966-4e28-9965-1de1fe4af490 service neutron] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:02:04.848369 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1469/2924] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 13:02:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:02:05.278226 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f7982-97e9-4342-92c4-1603acbf7b47 None None] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:02:05.279730 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-ce5f7982-97e9-4342-92c4-1603acbf7b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:02:05.354686 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f7982-97e9-4342-92c4-1603acbf7b47 None None] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:02:05.359568 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac8b2b-f5d6-413f-a8e4-2d6a7abb3171 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25eef64e92e9422ea62508c3eb0a197e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:02:05.360171 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac8b2b-f5d6-413f-a8e4-2d6a7abb3171 service placement] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:02:05.360448 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac8b2b-f5d6-413f-a8e4-2d6a7abb3171 service placement] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:02:05.360718 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac8b2b-f5d6-413f-a8e4-2d6a7abb3171 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:02:05.362432 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-bbac8b2b-f5d6-413f-a8e4-2d6a7abb3171 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:02:05.374595 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbac8b2b-f5d6-413f-a8e4-2d6a7abb3171 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:02:05.376186 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbac8b2b-f5d6-413f-a8e4-2d6a7abb3171 service placement] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:02:05.386659 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1456/2925] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 13:02:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:02:13.143092 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e652fc-7966-4e28-9965-1de1fe4af490 service neutron] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:02:13.145179 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-a1e652fc-7966-4e28-9965-1de1fe4af490 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:02:13.219931 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e652fc-7966-4e28-9965-1de1fe4af490 service neutron] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:02:13.227196 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a5f71b-251f-45a5-9c51-b29cef14541a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990f5837994f4a088c656c9b33e9a0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:02:13.227843 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a5f71b-251f-45a5-9c51-b29cef14541a service nova] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:02:13.228123 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a5f71b-251f-45a5-9c51-b29cef14541a service nova] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:02:13.228395 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a5f71b-251f-45a5-9c51-b29cef14541a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:02:13.230583 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-c1a5f71b-251f-45a5-9c51-b29cef14541a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:02:13.245611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a5f71b-251f-45a5-9c51-b29cef14541a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:02:13.247203 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a5f71b-251f-45a5-9c51-b29cef14541a service nova] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:02:13.258964 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1470/2926] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 13:02:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:02:15.381721 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=54 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:02:23.254677 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=83 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:02:44.812698 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac8b2b-f5d6-413f-a8e4-2d6a7abb3171 service placement] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:02:44.823354 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac8b2b-f5d6-413f-a8e4-2d6a7abb3171 service placement] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:02:44.828710 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29f336-8cbd-43f5-a092-9827f32595c0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25eef64e92e9422ea62508c3eb0a197e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:02:44.829308 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29f336-8cbd-43f5-a092-9827f32595c0 service placement] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:02:44.829586 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29f336-8cbd-43f5-a092-9827f32595c0 service placement] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:02:44.829885 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29f336-8cbd-43f5-a092-9827f32595c0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:02:44.831640 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-4e29f336-8cbd-43f5-a092-9827f32595c0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:02:44.843268 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e29f336-8cbd-43f5-a092-9827f32595c0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:02:44.844934 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e29f336-8cbd-43f5-a092-9827f32595c0 service placement] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:02:44.854761 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1457/2927] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 13:02:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:02:54.850628 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=6 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:03:10.652770 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f5c099-5f0a-4974-a3f9-8dc7baf7c5cb None None] REQUEST_METHOD: `POST` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:03:10.653316 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f5c099-5f0a-4974-a3f9-8dc7baf7c5cb None None] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:03:10.653434 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f5c099-5f0a-4974-a3f9-8dc7baf7c5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:03:10.673713 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.auth.core [None req-e4f5c099-5f0a-4974-a3f9-8dc7baf7c5cb None None] MFA Rules not processed for user `5a67a58ae60a4e13a3bfe77462c4337e`. Rule list: `[]` (Enabled: `True`). {{(pid=81806) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:03:10.674886 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-e4f5c099-5f0a-4974-a3f9-8dc7baf7c5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:03:10.681670 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1471/2928] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 13:03:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:03:10.920525 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29f336-8cbd-43f5-a092-9827f32595c0 service placement] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:03:10.929830 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29f336-8cbd-43f5-a092-9827f32595c0 service placement] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:03:10.933383 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97954bb-47cc-417a-9d2f-cac38a5ec05c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb327a8a18ca414ab8b4111a4d13aad8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:03:10.933975 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97954bb-47cc-417a-9d2f-cac38a5ec05c service neutron] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:03:10.934254 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97954bb-47cc-417a-9d2f-cac38a5ec05c service neutron] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:03:10.934524 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97954bb-47cc-417a-9d2f-cac38a5ec05c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:03:10.936225 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-d97954bb-47cc-417a-9d2f-cac38a5ec05c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:03:10.945676 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97954bb-47cc-417a-9d2f-cac38a5ec05c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:03:10.947079 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97954bb-47cc-417a-9d2f-cac38a5ec05c service neutron] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:03:10.955628 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1458/2929] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 13:03:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:03:20.667771 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=3 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:03:20.951414 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=6 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:04:14.106611 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f5c099-5f0a-4974-a3f9-8dc7baf7c5cb None None] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:04:14.118533 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f5c099-5f0a-4974-a3f9-8dc7baf7c5cb None None] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:04:14.123748 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968118a-cdb5-4a4b-9850-11002baac5b0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25eef64e92e9422ea62508c3eb0a197e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:04:14.129873 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968118a-cdb5-4a4b-9850-11002baac5b0 service placement] REQUEST_METHOD: `GET` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:04:14.129873 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968118a-cdb5-4a4b-9850-11002baac5b0 service placement] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:04:14.129873 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968118a-cdb5-4a4b-9850-11002baac5b0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:04:14.129873 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.fernet_utils [None req-2968118a-cdb5-4a4b-9850-11002baac5b0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81806) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:04:14.138533 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2968118a-cdb5-4a4b-9850-11002baac5b0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:04:14.140139 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2968118a-cdb5-4a4b-9850-11002baac5b0 service placement] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:04:14.150827 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1472/2930] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 13:04:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:04:14.249547 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97954bb-47cc-417a-9d2f-cac38a5ec05c service neutron] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:04:14.263215 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97954bb-47cc-417a-9d2f-cac38a5ec05c service neutron] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:04:14.267846 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01873d2a-ec9d-4b22-b392-6cba41dd4582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:04:14.268513 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01873d2a-ec9d-4b22-b392-6cba41dd4582 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:04:14.268790 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01873d2a-ec9d-4b22-b392-6cba41dd4582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:04:14.269764 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01873d2a-ec9d-4b22-b392-6cba41dd4582 admin admin] PATH_INFO: `/v3/users/5a67a58ae60a4e13a3bfe77462c4337e` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:04:14.270897 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01873d2a-ec9d-4b22-b392-6cba41dd4582 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a67a58ae60a4e13a3bfe77462c4337e)` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:04:14.272823 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01873d2a-ec9d-4b22-b392-6cba41dd4582 admin admin] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:04:14.454797 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-01873d2a-ec9d-4b22-b392-6cba41dd4582 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a67a58ae60a4e13a3bfe77462c4337e'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:04:14.467257 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-01873d2a-ec9d-4b22-b392-6cba41dd4582 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a67a58ae60a4e13a3bfe77462c4337e'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:04:14.468082 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-01873d2a-ec9d-4b22-b392-6cba41dd4582 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a67a58ae60a4e13a3bfe77462c4337e'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:04:14.475298 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-01873d2a-ec9d-4b22-b392-6cba41dd4582 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a67a58ae60a4e13a3bfe77462c4337e'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:04:14.549559 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.notifications [None req-01873d2a-ec9d-4b22-b392-6cba41dd4582 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a67a58ae60a4e13a3bfe77462c4337e'} {{(pid=81805) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:04:14.555690 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1459/2931] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 13:04:14 2022] DELETE /identity/v3/users/5a67a58ae60a4e13a3bfe77462c4337e => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:04:14.796531 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968118a-cdb5-4a4b-9850-11002baac5b0 service placement] Authenticating user token {{(pid=81806) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:04:14.811134 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968118a-cdb5-4a4b-9850-11002baac5b0 service placement] Validating token access rules against request {{(pid=81806) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:04:14.864225 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa771ed-9cbf-4ea9-8471-987cbf722e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77a14102256040ab81672f6adeaa6c5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '912de75d08d24473bf98cc746c209242', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81806) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:04:14.864909 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa771ed-9cbf-4ea9-8471-987cbf722e3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:04:14.865220 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa771ed-9cbf-4ea9-8471-987cbf722e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:04:14.865495 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa771ed-9cbf-4ea9-8471-987cbf722e3e admin admin] PATH_INFO: `/v3/projects/9421d499349f4d90b729021b00961cf2` {{(pid=81806) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:04:14.866667 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa771ed-9cbf-4ea9-8471-987cbf722e3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9421d499349f4d90b729021b00961cf2)` {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:04:14.868607 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa771ed-9cbf-4ea9-8471-987cbf722e3e admin admin] RBAC: Authorization granted {{(pid=81806) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:04:14.983304 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-2aa771ed-9cbf-4ea9-8471-987cbf722e3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9421d499349f4d90b729021b00961cf2', 'request_id': 'req-2aa771ed-9cbf-4ea9-8471-987cbf722e3e'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:04:14.990543 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-2aa771ed-9cbf-4ea9-8471-987cbf722e3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9421d499349f4d90b729021b00961cf2', 'request_id': 'req-2aa771ed-9cbf-4ea9-8471-987cbf722e3e'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:04:15.063737 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-2aa771ed-9cbf-4ea9-8471-987cbf722e3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9421d499349f4d90b729021b00961cf2', 'request_id': 'req-2aa771ed-9cbf-4ea9-8471-987cbf722e3e'} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:04:15.075654 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-2aa771ed-9cbf-4ea9-8471-987cbf722e3e admin admin] The token cache is being invalidate because project 9421d499349f4d90b729021b00961cf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81806) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:04:15.075948 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-2aa771ed-9cbf-4ea9-8471-987cbf722e3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:04:15.076602 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG keystone.notifications [None req-2aa771ed-9cbf-4ea9-8471-987cbf722e3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81806) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:04:15.078167 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: [pid: 81806|app: 0|req: 1473/2932] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 13:04:14 2022] DELETE /identity/v3/projects/9421d499349f4d90b729021b00961cf2 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:04:24.552155 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:04:25.072356 dv-cinder-844563-12-iscsi devstack@keystone.service[81806]: DEBUG dbcounter [-] [81806] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81806) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:07:46.781398 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01873d2a-ec9d-4b22-b392-6cba41dd4582 admin admin] Authenticating user token {{(pid=81805) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:07:46.782917 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-01873d2a-ec9d-4b22-b392-6cba41dd4582 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:07:46.920836 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01873d2a-ec9d-4b22-b392-6cba41dd4582 admin admin] Validating token access rules against request {{(pid=81805) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:07:46.979214 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c9c4a7-a23b-4eef-b5cd-ba7929149b35 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25eef64e92e9422ea62508c3eb0a197e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea2fa8ad01240f7859182ebbe3051c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81805) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:07:46.979852 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c9c4a7-a23b-4eef-b5cd-ba7929149b35 service placement] REQUEST_METHOD: `GET` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:07:46.980132 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c9c4a7-a23b-4eef-b5cd-ba7929149b35 service placement] SCRIPT_NAME: `/identity` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:07:46.980407 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c9c4a7-a23b-4eef-b5cd-ba7929149b35 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81805) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:07:46.982122 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.fernet_utils [None req-15c9c4a7-a23b-4eef-b5cd-ba7929149b35 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81805) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:07:46.993315 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c9c4a7-a23b-4eef-b5cd-ba7929149b35 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:07:46.994880 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c9c4a7-a23b-4eef-b5cd-ba7929149b35 service placement] RBAC: Authorization granted {{(pid=81805) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:07:47.005132 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: [pid: 81805|app: 0|req: 1460/2933] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 13:07:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:07:57.000524 dv-cinder-844563-12-iscsi devstack@keystone.service[81805]: DEBUG dbcounter [-] [81805] Writing DB stats keystone:SELECT=89 {{(pid=81805) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}